Video signal processing device and method, image synthesizing device, and editing device

ABSTRACT

In synthesizing a foreground picture and a background picture in accordance with a key signal, for example, by soft chroma key processing, a preview screen  102  as a picture as a result of processing by the key signal and a key parameter setting screen  101  for setting characteristics of the key signal are displayed. Characteristics of the key signal are changed on a three-dimensional space by information obtained through the key parameter setting screen  101 , and the picture as a result of processing is changed on the preview screen  102 . Thus, even when the range of key signal setting is adjusted on the three-dimensional color space, desired processing may be carried out by simple operation.

TECHNICAL FIELD

This invention relates to an image signal processing device and method,a picture synthesizing device, and an editing device which are used forpicture synthesis employing a so-called chroma key.

BACKGROUND OF THE INVENTION

Conventionally, in chroma key processing using an editing device at abroadcasting station or the like, picture synthesis is carried out byinserting a background picture into a foreground picture.

Specifically, at a broadcasting station or the like, for example, anannouncer is imaged with a screen of blue color as a background(so-called blue back), thereby generating an image signal of aforeground picture. Also, a desired object to be allocated to thebackground of the foreground picture is imaged, thereby generating animage signal of a background picture.

In an editing device or the like, in response to the operation by anoperator, a color to be extracted from the foreground picture is presetby setting a reference signal level with respect to color-differencesignals. In the case where picture synthesis is to be carried out byusing a soft chroma key, in the editing device, lower limit and upperlimit signal levels (that is, threshold values) are set with referenceto a center color of the color to be extracted.

The editing device generates a key signal by sequentially comparing thethreshold values with the image signal of the foreground picture.Specifically, with respect to the foreground picture generated with thebackground of blue color screen, the editing device generates a keysignal by setting the upper limit and lower limit threshold values sothat a value 0 is obtained in the background part while a value 1 isobtained with colors other than the blue color of the background. Withrespect to other bluish colors, a key signal is generated within a rangeof the value 1 to the value 0 corresponding to the upper limit and lowerlimit threshold values so that a value corresponding to the degree ofblueness is obtained.

Thus, the conventional editing device generates a key signal bysequentially discriminating the color of the foreground picture, using atwo-dimensional color space expressed by color-difference signals (thatis, a color space with UV signals as coordinate axes). The editingdevice synthesizes the foreground picture and the background picturewith reference to the key signal thus generated, so as to generate asynthetic picture such that the background picture is inserted in thebackground of the foreground picture.

Meanwhile, in the conventional editing device, it is difficult todiscriminate a portion of a high luminance level and a portion of a lowluminance level with the same color. Therefore, with respect to asynthetic picture generated by the conventional editing device, therearises a problem such that the contour of the inserted backgroundpicture is displayed in an unnatural manner, thus lowering thedefinition in comparison with the case where no chroma key processing iscarried out.

As a method for solving this problem, it may be considered to expressand process each pixel of the foreground picture on a three-dimensionalcolor space. In this method, however, the range of key signal settingmust be adjusted on the three-dimensional color space instead of theconventional two-dimensional color space, and therefore the operation toadjust chroma key processing might be complicated.

DISCLOSURE OF THE INVENTION

In view of the foregoing problem, it is an object of the presentinvention to provide an image signal processing device and method, apicture synthesizing device, and an editing device which enable picturesynthesis of high definition with a simple structure and enable desiredprocessing by simple operation.

In an image signal processing device and method according to the presentinvention, in generating a key signal for synthesizing a foregroundpicture and a background picture, a polar coordinate of each pixel ofthe foreground picture is detected in a three-dimensional color spacehaving a center color of a color to be extracted from the foregroundpicture as an origin, and the key signal is generated in accordance withthe distance of the polar coordinate from the origin.

Also, in generating a key signal for synthesizing a foreground pictureand a background picture, the value of the key signal is set inaccordance with the distance from a predetermined reference color in athree-dimensional color space to each pixel of the foreground picture,and a picture as a result of processing by the key signal and acharacteristic setting screen for the key signal are displayed. The n,characteristics of the key signals are changed by information obtainedthrough the characteristic setting screen, and the picture as a resultof processing is changed.

In addition, in generating a key signal for synthesizing a foregroundpicture and a background picture, the value of the key signal is set inaccordance with the distance of each pixel of the foreground picturefrom a reference center color in a three-dimensional color space, andposition information of a pixel designated in the foreground picturefrom a predetermined reference position in the three-dimensional colorspace.

Also, in generating a key signal for synthesizing a foreground pictureand a background picture, a pixel forming the foreground picture isprojected on a reference plane set on a three-dimensional color space,thus displaying the position of the pixel forming the foreground pictureon the three-dimensional color space.

In addition, in generating a key signal for synthesizing a foregroundpicture and a background picture, the value of the key signal is set inaccordance with the distance from a predetermined reference color in athree-dimensional color space, and each pixel of at least the foregroundpicture is located at a corresponding position on the three-dimensionalcolor space, thus displaying a picture viewed from a desired viewpoint.

The key signal generating, device and method having such characteristicsmay be applied to a picture synthesizing device based on the frame unitand an editing device for carrying out editing of a plurality of frames.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the schematic structure of an editingdevice to which an embodiment of the present invention is applied.

FIG. 2 is a block diagram showing an example of a key signal generatingdevice as an embodiment of the present invention.

FIGS. 3A and 3B show the relations between a foreground picture and acenter color.

FIG. 4 illustrates setting of conditions of chroma key processing.

FIGS. 5A and 5B show a boundary and a chroma key signal on athree-dimensional color space of chroma key processing.

FIG. 6 shows a group of points specifying the boundary on thethree-dimensional color space of chroma key processing.

FIGS. 7A and 7B show a boundary and a color cancel key signal on atwo-dimensional color space of color cancel processing.

FIG. 8 shows a group of points specifying the boundary on thetwo-dimensional color space of color cancel processing.

FIG. 9 illustrates calculation of the distance from the group of pointsof FIG. 6 to a boundary of a greater diameter.

FIG. 10 illustrates calculation of the distance from the group of pointsof FIG. 6 to a boundary of a smaller diameter.

FIGS. 11A and 11B show the relations between the boundary of FIG. 5 andcharacteristics of chroma key processing.

FIG. 12 is a block diagram showing a chroma key processing section ofthe editing device of FIG. 1.

FIG. 13 illustrates processing of a foreground picture.

FIG. 14 illustrates a boundary on the three-dimensional color space ofchroma key processing.

FIGS. 15A and 15B show the relations between a boundary on thetwo-dimensional color space of color cancel processing and a key signalfor color cancel.

FIGS. 16A to 16F illustrate picture synthesis which employs add-mixsynthesis.

FIG. 17 is a flowchart showing procedures for setting chroma keyprocessing conditions.

FIG. 18 shows a preview screen in the editing device of the firstembodiment of the present invention.

FIG. 19 shows a specific example of the preview screen.

FIG. 20 illustrates a key parameter setting screen.

FIG. 21 shows a specific example of the key parameter setting screen.

FIG. 22 shows a center color setting section of the key parametersetting screen.

FIG. 23 is a flowchart for illustrating processing of a pixel in aforeground picture.

FIG. 24 shows the relations between luminance, color difference, angleand distance in the processing procedures of FIG. 23.

FIG. 25 shows a vector scope display section.

FIG. 26 illustrates a first display section of the vector scope displaysection.

FIG. 27 illustrates a second display section of the vector scope displaysection.

FIG. 28 illustrates a third display section of the vector scope displaysection.

FIG. 29 illustrates a fourth display section of the vector scope displaysection.

FIG. 30 is a flowchart showing display processing in the first to thirddisplay sections.

FIG. 31 illustrates the processing procedures of FIG. 30. FIG. 32 showsan example of a color space display section.

FIG. 33 illustrates switching of a viewpoint in the color space displaysection.

FIG. 34 illustrates rotation of a color space in the color space displaysection.

FIG. 35 is a flowchart showing processing procedures of a centralprocessing unit of a second embodiment.

FIG. 36 illustrates processing for changing a representative point inanother embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

Preferred embodiments of the present invention will now be describedwith reference to the drawings.

First, an example of an editing device to which an embodiment of the keysignal generating device of the present invention is applied will bedescribed with reference to FIG. 1.

FIG. 1 is a block diagram showing the schematic structure of an editingdevice 1 for carrying out picture synthesis by chroma key processing andcolor cancel processing.

The editing device 1 shown in FIG. 1 is constituted by arranging apicture processing board onto a computer. The editing device 1 isadapted for recording various image signals onto a hard disk drive (HDD)2 in response to the operation by an operator, then editing the recordedimage signals, and outputting the edited image signals to an externalequipment. In this case, the editing device 1 synthesizes image signalsSV1 and SV2 of a foreground and a background recorded on the hard diskdrive 2 so as to generate an image signal SV3 of a synthetic picture,and records this image signal SV3 onto the hard disk drive 2.

Specifically, the hard disk drive 2 switches the operation in responseto a control command inputted through a bus BUS, so as to record animage signal necessary for editing of the editing device 1 and outputthe recorded image signal. Frame buffers 3 and 4 switch the operation inresponse to control data similarly inputted through a bus BUS, so as tostore the image signals SV1 and SV2 sequentially inputted thereto,respectively, and output the image signals at a predetermined timing. Achroma key processing section 5 synthesizes the image signals SV1 andSV2 inputted from the frame buffers 3 and 4 by a chroma key method, andoutputs the image signal SV3 constituting the synthetic picture.Therefore, in the chroma key processing section 5, a key signalgenerating section 6 generates a key signal from the image signal SV1 ofthe foreground picture, and a picture synthesizing section 7 generatesthe image signal SV3 by this key signal.

As the key signal generating section 6 in the chroma key processingsection 5, a structure as shown in FIG. 2 as an embodiment of thepresent invention may be used. The key signal generating section 6 ofFIG. 2 will be later described in detail.

Referring to FIG. 1 again, a frame buffer 8 stores the image signal SV3outputted from the chroma key processing section 5, and outputs theimage signal SV3 to the hard disk drive 2 and a monitor 9 at apredetermined timing. Thus, in the editing device 1, the image signalSV3 chroma key processed by the chroma key processing section 5 isconfirmed on the monitor 9, and recorded onto the hard disk drive 2. Inthe editing device 1, the frame buffers 3, 4, 8 and the chroma keyprocessing section 5 are constituted on the board, and this board isconnected to a bus BUS of the computer.

A central processing unit (CPU) 10 secures a work area in a randomaccess memory (RAM) 11, and carries out a series of processingprocedures stored on a read-only memory (ROM) 14 and the hard disk drivein response to the operation on a keyboard 12 and a mouse 13. Thus, thecentral processing unit 10 prepares an editing list necessary forprocessing of this editing device and controls the entire operation inaccordance with the prepared editing list, thereby carrying out a seriesof editing processing in accordance with the operation by the operator.

In this series of processing, when the operator selects editing bychroma key, the central processing unit 10 starts the operation of thechroma key processing section 5 and the frame buffers 3, 4, 8, so as tocarry out chroma key processing. In this processing, after acceptingconditions setting of chroma key processing, the central processing unit10 carries out parameter setting processing of the chroma key processingsection 5 in accordance with the accepted conditions, and starts chromakey processing by the set parameter.

Specifically, in the processing of condition setting, the centralprocessing unit 10 displays a still picture of the foreground picture onthe monitor 9, and when the operator moves a cursor on the monitor andclicks the mouse 13, the central processing unit 10 sets a correspondingpixel to a pixel of a center color. The center color is a referencecolor of a portion to be extracted from the foreground picture. In thisediting device 1, the hue and luminance of the foreground picture arediscriminated with reference to a predetermined area on athree-dimensional color space formed around the center color, and a keysignal is generated on the basis of the result of discrimination.

When the operator designates a pixel C0 on a foreground picture MF asshown in FIG. 3A, the central processing unit 10 detects the luminanceY0 and the hues U0, V0 of this pixel, and sets a color specified by theluminance Y0 and the hues U0, V0 to a center color C0 as shown in FIG.3B. In addition, the central processing unit 10 sets double sphericalshape K1 and K2 centering the center color C0 on a three-dimensionalcolor space with luminance Y and color differences U, V as referenceaxes. At this point, the central processing unit 10 sets a presetreference value as a radius so as to form the two spherical shapes K1and K2.

Also, the central processing unit 10 carries out processing partlycommon to chroma key processing as later described, thereby displayingan area of a color included inside of the spheres K1, K2 with respect tothe still picture displayed on the monitor 9. In this state, the centralprocessing unit 10 partly transforms the outer spherical shape K2, forexample, as indicated by an arrow A in FIG. 4, in response to theoperation inputted by the operator through the display on the monitor 9.This transformation is similarly carried out with respect to the innerspherical shape K1 in response to the operation by the operator.

Using the two spherical shapes K1, K2 thus set on the color space, thecentral processing unit 10 generates a key signal KEY of a value 0 so asto allocate the background with respect to a color included in an innerarea AR1 of the spherical shape K1 of a smaller diameter, as shown inFIG. 5A. Also, with respect to a color located in an outer area AR3 ofthe sphere K2 of a greater diameter, the central processing unit 10generates a key signal KEY of a value 1 so as to allocate theforeground. In addition, with respect to a color included in an area AR2between spheres K1 and K2, the central processing unit 10 generates akey signal KEY of a value corresponding to the position of the colorwithin a range of values 1 to 0. FIG. 5B shows an example of the keysignal KEY. Thus, the central processing unit 10 sets boundaries K1 andK2 for switching the characteristics of the key signal KEY in accordancewith the substantially spherical shapes K1 and K2 set by the operator.

As shown in FIG. 6, the central processing unit 10 expresses theboundaries K1 and K2 thus set, by using 26 points (hereinafter referredto as representative points) P0 to P25, respectively. In addition, thecentral processing unit 10 expresses the representative points P0 to P25by coordinate values expressed on a polar coordinate, on a Y, U, Vspatial coordinate with the center color C0 as an origin. Specifically,the central processing unit 10 generates a plurality of vectors atconformal 45 degrees in a latitudinal direction (a direction from thecenter color C0 prescribed by an angle A) and in a longitudinaldirection (a direction from the center color C0 prescribed by an angle6) in the polar coordinate space, and expresses the representativepoints P0 to P25 by the length r of the vectors. Therefore, in the casewhere the boundaries K1 and K2 are to be set by the man-machineinterface as described above, the central processing unit 10 varies thelength r of the corresponding vector in accordance with the operation bythe operator. The angle at the time of expressing the representativepoint is not limited to 45 degrees and may be 22.5 degrees or otherarbitrary angles.

Moreover, the central processing unit 10 cuts the spherical shapes K1,K2 on a plane perpendicular to a Y-axis crossing the center color C0 soas to form double circular shape K1C, K2C as shown in FIG. 7A. Thecentral processing unit 10 transforms the circular shape K1C, K2C inresponse to the operation by the operator, and displaces the center C0C(U0C, V0C) of the circular shapes K1C, K2C. Thus, the central processingunit 10 sets a range for color cancel by the double circular shapes K1C,K2C, and sets two boundaries for switching characteristics with respectto a color cancel key signal CCK. Specifically, as shown in FIG. 7B,with respect to an area ARIC on the inner side of the circular shape K1Cof a smaller diameter, CCK=1 is set so as to carry out color cancel.With respect to an area AR3C on the outer side of the circular shape K2Cof a greater diameter, CCK=0 is set so as not to carry out color cancel.With respect to an area AR2C between the circular shape K1C and K2C, acolor cancel key signal CCK within a range of values 1 to 0 is generatedso as to carry out color cancel corresponding to the position.

With respect to such range for color cancel, the central processing unit10 sets vectors at a conformal spacing of 45 degrees as shown in FIG. 8,on the basis of the coordinate (U0C, V0C) of the center C0C changed bythe operation by the operator as the origin, and expresses the circularshapes by a group of eight representative points P0C to P7C prescribedby the length of the vectors. This angle may also be set to an arbitraryangle such as 22.5 degrees. Therefore, in the case where the boundariesK1C and K2C of the color cancel range are to be set by theabove-described man-machine interface, the central processing unit 10varies the length rC of the corresponding vector in accordance with theoperation by the operator.

The technique of color cancel will now be described. Basically, inchroma key synthesis, the edge of an object or a semi-transparentportion like smoke or a glass is more or less colored with a backgroundcolor (for example, blue at the time when a blue back is used), and thiscoloring is prevented by the color cancel function. In this technique,after the same color as the back color in the foreground picture ischanged to be colorless, synthesis is carried out.

In principle, color cancel may be carried out by removing the same coloras the back color from the foreground picture, that is, adding acomplementary color. However, if this processing is carried out on theentire foreground picture, the color of the object is also changed.Therefore, an area where color cancel is to be carried out must bedesignated. The area is designated by the color cancel signal CCK. Thiscolor cancel signal CCK is proximate to a signal obtained by invertingthe key signal KEY for chroma key. However, since the method forgenerating the original key signal differs, the inverted color cancelsignal CCK is not perfectly identical to the key signal KEY for chromakey.

When the boundaries K1, K2 for chroma key and the boundaries K1C, K12Cfor color cancel are thus set in accordance with the operation by theoperator, the central processing unit 10 shifts to parameter settingprocessing. The central processing unit 10 calculates a parameter to beset to the chroma key processing section 5, and then sets this parameterto the chroma key processing section 5.

In calculation of this parameter, the central processing unit 10calculates a parameter of chroma key processing by the boundaries K1,K2. Specifically, as shown in FIGS. 9 and 10, the central processingunit 10 sequentially changes the latitude φ and the longitude θ on theY, U, V color space based on the center color CO described above in FIG.6, and calculates the lengths ri and ro to points RI and RO where avector V extending from the origin crosses the boundaries K1 and K2,respectively, in each latitude φ and longitude θ.

At this point, the central processing unit 10 sequentially calculatesthe lengths ri and ro by interpolation processing of the followingEquations (1) and (2) using the coordinate of adjacent fourrepresentative points with respect to each of the boundaries K1 and K2.$\begin{matrix}{{{ro}\left( {\varphi,\theta} \right)} = \frac{SO}{\left( {{\theta 1} + {\theta \quad 2}} \right) \cdot \left( {{\varphi 1} + {\varphi \quad 2}} \right)}} & (1)\end{matrix}$

SO=roa·θ2·φ2+rob·θ1·φ2+roc·θ2·100 1+rod·θ1·φ1 $\begin{matrix}{{{ri}\left( {\varphi,\theta} \right)} = \frac{SI}{\left( {{\theta 1} + {\theta \quad 2}} \right) \cdot \left( {{\varphi 1} + {\varphi \quad 2}} \right)}} & (2)\end{matrix}$

SI=ria·θ2·φ2+rib·θ1·φ2+ric·θ2·φ1+rid·θ1·φ1

In addition, the central processing unit 10 calculates a clip point CLIPand a gain GAIN from the lengths ri and ro to the points RI and RO thusdetected with respect to the latitude φ and the longitude θ, as shown inFIGS. 11A and 11B. The central processing unit 10 sets the length ricalculated from the boundary K1 of the smaller diameter to the clippoint CLIP. The central processing unit 10 sets the clip point CLIP to avalue 0, and sets the position of the length ro calculated from theboundary K2 of the greater diameter to a value 1. The central processingunit 10 then sets the slope of a straight line connecting the clip pointCLIP and the point of the value 1 to the gain GAIN.

That is, the central processing unit 10 carries out processing of thefollowing equations so as to calculate the clip point CLIP and the gainGAIN for every latitude φ and longitude θ.

CLIP(φ, θ)=ri(φ,θ)  (3)

$\begin{matrix}{{{GAIN}\left( {\varphi,\theta} \right)} = \frac{1.0}{{{ro}\left( {\varphi,\theta} \right)} - {{ri}\left( {\varphi,\theta} \right)}}} & (4)\end{matrix}$

Thus, the central processing unit 10 sets a characteristic curve of keysignal generation in accordance with the clip point CLIP and the gainGAIN for every latitude φ and longitude θ. This characteristic curve isheld at the value 0 up to the boundary K1 of the smaller diameter, andis expressed by connecting the value 0 to the value 1 between theboundary K1 of the smaller diameter and the boundary K2 of the greaterdiameter, in accordance with the distance r from the center color COwith respect to each latitude φ and longitude θ.

Having thus calculated the clip point CLIP and gain GAIN, the centralprocessing unit 10 sets the clip point CLIP and the gain GAIN to a keysignal generating section 6, and forms a look-up table of the clip pointCLIP and the gain GAIN using each latitude φ and longitude θ as anaddress, in the key signal generating section 6.

Similarly, the central processing unit 10 calculates a clip point and again for every latitude with respect to the range for color cancel. Inaddition, the central processing unit 10 sets the calculated clip pointand gain to the key signal generating section 6, and forms a look-uptable of the clip point and the gain using each longitude θ as anaddress, in the key signal generating section 6.

Having thus formed the look-up table in the key signal generatingsection 6, the central processing unit 10 completes the parametersetting processing.

FIG. 12 is a block diagram showing, the picture synthesizing section 7.The picture synthesizing section 7 inputs a luminance signal Y1 andcolor-difference signals U1, V1 constituting the image signal SV1, and aluminance signal Y2 and color-difference signals U2, V2 constituting theimage signal SV2, in the form of digital signals. Color cancel circuits20Y, 20U, 20V receive the luminance signal Y and the color-differencesignals U1, V1 of the image signal of the foreground picture,respectively, and carry out color cancel processing as described aboveon the luminance signal Y and the color-difference signals U1, V1 by thecolor cancel key signal CCK so as to output the color-cancelled signals.

The color cancel key signal CCK is generated by the key signalgenerating section 6 with reference to the above-described color cancelrange. Thus, the color cancel circuits 20Y, 20U, 20V carry out colorcancel processing on a contour portion which is left by chroma keyprocessing, so as to effectively prevent reflection of the backgroundonto the foreground picture.

Keyers 21Y, 21U, 21V carry out weighting by a key signal KEY on theluminance signal Y1 and the color-difference signals U1, V1 outputtedfrom the color cancel circuits 20Y, 20U, 20V, so as to output theweighted signals. Keyers 22Y, 22U, 22V receive the luminance signal Y2and the color-difference signals U2, V2 of the image signal of thebackground picture, and carry out weighting by a key signal 1−KEY on theluminance signal Y2 and the color-difference signals U2, V2 so as tooutput the weighted signals. The key signal 1−KEY inputted to the keyers22Y, 22U, 22V is generated by the key signal generating section 6 withreference to the key signal KEY so that the key signal 1−KEY added tothe key signal KEY inputted to the keyers 21Y, 21U, 21V forms aweighting value of 1.

Mixers 23Y, 23U, 23V adds the luminance signal Y1 and thecolor-difference signals U1, VI outputted from the keyers 21Y, 21U, 21Vto the luminance signal Y2 and the color-difference signals U2, V2outputted from the keyers 22Y, 22U, 22V, and output an image signal SV3by a luminance signal Y3 and color-difference signals U3, V3.

Thus, the picture synthesizing section 7 synthesizes the foreground andthe background picture by the key signals KEY and 1KEY, and at the sametime, reduces reflection of the background onto the foreground pictureby the key signal CCK.

Referring to FIG. 2, a key signal generating device as a firstembodiment of the present invention will now be described.

FIG. 2 is a block diagram showing the key signal generating device,which is the first embodiment of the present invention, as an example ofthe key signal generating section 6 of FIG. 1. In the key signalgenerating section 6, a chroma key signal generating section 25 inputscolor-difference signals U1, V1 to over-sampling circuits 26U, 26V,respectively, so as to over-sample the color-difference signals U1, V1by the same sampling frequency as for a luminance signal Y. Thus, thechroma key signal generating section 25 effectively prevents generationof noise in the subsequent processing of the color-difference signal U1,V1.

A coordinate transformation circuit 27 receives the color-differencesignals U1, V1, and specifies each pixel of the image signal SV1 of thecolor-difference signals U1, V1 by polar coordinate on a UV plane withthe center color CO as the origin. That is, the coordinatetransformation circuit 27 subtracts color-difference signal levels U0,V0 of the center color CO from the color-difference signal levels U1,V1, respectively. From the result of subtraction, the coordinatetransformation circuit 27 calculates an angle θ on the UV plane and adistance rm from the origin CO.

The coordinate transformation circuit 27 carries out arithmeticprocessing of the following equations, thereby detecting the length rmof a line formed by projecting onto the UV plane a line L connecting theposition of each pixel of the image signal SV1 on a YUV space with thecenter color CO as the origin and the origin CO, and the longitude θ ofthe line L on the YUV space, as shown in FIG. 13. $\begin{matrix}{\theta = {\arctan \quad \frac{{V1} - {V0}}{{U1} - {U0}}}} & (5)\end{matrix}$

rm=((U1−U0)²+(V1−V0)²)^(½)  (6)

A coordinate transformation circuit 28 carries out arithmetic processingof the following equations from the distance rm calculated by thecoordinate transformation circuit 27 and the luminance signal level Y1,thereby calculating the latitude φ and the distance rM to the origin COwith respect to the position of each pixel of the image signal SV1 onthe YUV space with the center color CO as the origin. $\begin{matrix}{\varphi = {\arctan \quad \frac{{Y1} - {Y0}}{rm}}} & (7)\end{matrix}$

rM=((Y1−Y0)²+rm²)^(½)  (8)

A look-up table 29 is formed by storing a clip point CLIP(x, y) and again GAIN(x, y) at each latitude x and longitude y calculated by thecentral processing unit 10 in advance. The look-up table 29 outputscorresponding clip point CLIP(φ, θ) and gain GAIN(φ, θ) using thelongitude θ and the latitude φ calculated by the coordinatetransformation circuits 27, 28. Thus, the look-up table 29 selectivelyoutputs a characteristic corresponding to each pixel of the image signalSV1, that is, a characteristic shown in FIG. 11B, from thecharacteristics of each latitude 4 and longitude θ calculated in advanceby the central processing unit 10.

A key process circuit 30 generates a key signal MXK corresponding to thedistance rM calculated by the coordinate transformation circuit 28, onthe basis of the clip point CLIP and the gain GAIN outputted from thelook-up table 29. Specifically, the key process circuit 30 outputs thekey signal MXK of a value 0 if the distance rM is shorter than the clippoint CLIP, that is, if the pixel of the image signal SV is located onthe inner side of the boundary K1 of the smaller diameter as indicatedby A in FIG. 14, as expressed by the following equation.

MXK=0(rM<CLIP)  (9)

On the other hand, if the distance rM is longer than the clip pointCLIP, the key process circuit 30 carries out arithmetic processing ofthe following Equation (10) and clips the result of this arithmeticprocessing to a value 1 for output.

The key process circuit 30 carries out arithmetic processing of thefollowing equations so as to generate the key signal MXK, depending uponthe case where the pixel of the image signal SV1 is located between theboundary K1 of the smaller diameter and the boundary K2 of the greaterdiameter (indicated by B in FIG. 14) or the case where the pixel islocated on the outside of the boundary K2 of the greater diameter(indicated by C in FIG. 14).

MXK=(rM−CLIP)×GAIN(CLIP<rM<ro)  (10)

MXK=1(ro<rM)  (11)

Thus, the key process circuit 30 in the chroma key signal generatingsection 25 generated the key signal MXK of chroma key with reference tothe luminance signal Y1 and the color-difference signals U1, V1 of theimage signal SV1 by using the preset look-up table 29.

Generation of the key signal for color cancel will now be described.

The key signal generating section 6 generates a key signal AMK for theabove-described color cancel, and inverts (subtracts from 1) this keysignal to provide a color cancel key signal CCK. The key signalgenerating section 6 also corrects the signal level of the key signalMXK for chroma key by the key signal AMK to provide a chroma key signalKEY. The picture synthesizing section 7 carries out color cancelprocessing using the color cancel signal CCK obtained by inverting(subtracting from 1) the key signal AMK, and carries out chroma keyprocessing using the corrected chroma key signal KEY, thereby carryingout picture synthesis in which the above-described reflection of thebackground color onto the contour portion of the object is reduced.

Specifically, in the key signal generating section 6, a color cancel keysignal generating section 32 inputs the color-difference signals U1, V1outputted from the over-sampling circuits 26U, 26V to a coordinatetransformation circuit 33. The coordinate transformation circuit 33detects the position of each pixel of the image signal SV1 on a UV planewith the center coordinate U0C, V0C used for the above-describedgeneration of the color cancel parameter. In this detection processing,the same arithmetic processing as the arithmetic processing of Equations(5) and (6) is carried out to calculate a distance rC and an angle θC.

A look-up table 34 is formed by storing a clip point CLIPC(x) and a gainGAINC(x) at each angle x calculated by the central processing unit 10 inadvance. The look-up table 34 outputs corresponding clip point CLIPC(φC)and gain GAINC(φC) using the angle OC calculated by the coordinatetransformation circuit 33. Thus, the look-up table 34 selectivelyoutputs a characteristic corresponding to each pixel of the image signalSV1 from the characteristics of each angle θC calculated in advance bythe central processing unit 10.

FIGS. 15A and 15B show the relations between the lengths riC, roC to thedouble circular shapes K1C, K2C of FIGS. 7A and 7B and the key signalAMK. On the assumption that the length riC from the center COC as areference color of color cancel to the boundary K1C of the smallercircular shape is the clip point CLIPC, the key signal AMK correspondingto the distance rMC to the center COC with respect to the position ofeach pixel of the image signal on the UV space is expressed by thefollowing Equations (12) to (14).

AMK=0(rMC<CLIPC)  (12)

AMK=(rMC−CLIPC)×GAINC(CLIPC<rMC<roC)  (13)

AIMK=1(roC<rMC)  (14)

A key process circuit 35 generates a key signal AMK corresponding to thedistance rC calculated by the coordinate transformation circuit 33, onthe basis of the clip point CLIPC(θC) and the gain GAINC(θC) outputtedfrom the look-up table 34, similarly to the key process circuit 30. Aninversion circuit 36 subtracts this key signal AMK from a value 1 so asto output a color cancel signal CCK. Thus, the color cancel key signalgenerating section 32 generates the key signal AMK corresponding to atwo-dimensional area set for color cancel by the operator.

In the chroma key signal generating section 25, a subtraction circuit 37subtracts, from a value 1, a key signal KEY for chroma key outputtedfrom a selection circuit 38 so as to output a key signal 1−KEY forbackground. The selecting circuit 38 generates the key signal KEY forchroma key from the key signals MXK, AMK outputted from the key processcircuits 30, 35. At this point, the selection circuit 38 switches theoperation interlocked with the color cancel key signal generatingsection 32 in response to the operation by the operator, therebycarrying out chroma key processing desired by the user.

In the case where the image signal SV3 outputted from the mixers 23Y,23U, 23V of FIG. 12 is expressed by the luminance signal Y and thecolor-difference signals U, V and where the operator does not selectcolor cancel processing, the selection circuit 38 outputs the key signalMXK outputted from the key process circuit 30, as the key signal KEY forchroma key, and the color cancel key signal generating section 32outputs the key signal CCK of a value 1. Thus, the chroma key processingsection 5 carries out chroma key processing expressed by the followingrelational expression. $\begin{matrix}{\begin{bmatrix}{Y3} \\{U3} \\{V3}\end{bmatrix} = {{{MXK} \cdot \begin{bmatrix}{Y1} \\{U1} \\{V1}\end{bmatrix}} + {\left( {1 - {MXK}} \right) \cdot \begin{bmatrix}{Y2} \\{U2} \\{V2}\end{bmatrix}}}} & (15)\end{matrix}$

In the case where the operator selects color cancel processing in afirst operation mode, the selection circuit 38 outputs the key signalMXK outputted from the key processing circuit 30, as the key signal KEYfor chroma key, and the color cancel key signal generating section 32outputs the key signal 1−AMK (=CCK) generated by the above-describedarithmetic processing, only with respect to the color-differencesignals. Thus, the chroma key processing section 5 carries out chromakey processing expressed by the following relational expression.$\begin{matrix}{\begin{bmatrix}{Y3} \\{U3} \\{V3}\end{bmatrix} = {{{MXK} \cdot \begin{bmatrix}{Y1} \\{{U1} - {{CCK} \cdot {U0C}}} \\{{V1} - {{CCK} \cdot {V0C}}}\end{bmatrix}} + {\left( {1 - {MXK}} \right) \cdot \begin{bmatrix}{Y2} \\{U2} \\{V2}\end{bmatrix}}}} & (16)\end{matrix}$

In this relational expression, U0C, V0C represent reference colors forcolor cancel preset by the operator.

In this processing of the first operation mode, color-difference signalcomponents U0C, V0C of the color of the center C0C of color cancel aresubtracted from the color-difference signal components U1, V1, that is,complementary colors are added in order to prevent the edge of theobject or a semi-transparent portion like smoke or a glass from becomingbluish (in the case of blue back). Thus, the same color as thebackground color (blue in the case of blue back) in the foregroundpicture is changed to be colorless. However, if this color cancelprocessing is carried out on the entire foreground picture, the color ofthe object is also changed. Therefore, an area where color cancel is tobe carried out must be designated. The color cancel key signal CCK is asignal for carrying out such designation.

In the case where the operator selects color cancel processing in asecond operation mode, the selection circuit 38 outputs the key signalAMK for color cancel as the key signal KEY, and the color cancel keysignal generating section 32 outputs the key signal 1−AMK (=CCK)generated by the above-described arithmetic processing. Thus, the chromakey processing section 5 carries out chroma key processing expressed bythe following relation expression. $\begin{matrix}{\begin{bmatrix}{Y3} \\{U3} \\{V3}\end{bmatrix} = {\begin{bmatrix}{Y1} \\{U1} \\{V1}\end{bmatrix} - {{CCK} \cdot \begin{bmatrix}{Y0C} \\{U0C} \\{V0C}\end{bmatrix}} + {\left( {1 - {MXK}} \right) \cdot \begin{bmatrix}{Y2} \\{U2} \\{V2}\end{bmatrix}}}} & (17)\end{matrix}$

In this relational expression, Y0C represents the luminance level of areference color for color cancel preset by the operator. Such switchingof the operation of the color cancel key signal generating section 32 iscarried out by switching the operation of the inversion circuit 36.

This second operation mode is referred to as an add-mix synthesis mode.In this synthesis, keying processing on the foreground picture is notcarried out. That is, with respect to the foreground picture, colorcancel is carried out only in a portion where the color is to becancelled, and the background picture on which keying processing hasbeen carried out is added so as to carry out synthesis. This processingis effective in the case where the operator wishes to carry out colorcancel while leaving all the information including the shadow of theobject or other noise generated on the blue back of the foregroundpicture.

FIGS. 16A to 16F illustrate such processing in the second operation mode(add-mix synthesis mode).

FIG. 16A shows an example of a foreground picture in which an object SBsuch as a person is located in front of a blue back BB as a background,with a bright portion BR and a shadow portion SH generated near theobject SB. FIG. 16B shows a luminance signal Y1 on a predeterminedhorizontal line g of an image signal of this foreground picture. FIG.16C shows the key signal MXK. With respect to the image signal keyed bythis key signal MXK, for example, the luminance signal, all theinformation including the shadow portion SH and the bright portion BR onthe blue back BB is cancelled, and only the information of the object SBremains, as shown in FIG. 16D. The color cancel key signal CCK is asignal which is narrower on the side of the object area than the signalobtained by inverting (subtracting from 1) the key signal MXK, as shownin FIG. 16E. FIG. 16F shows a signal (luminance signal component of theimage signal) obtained by calculating Y1−CCK·Y0C for the above-describedadd-mix synthesis. The same can be applied to the color-differencesignal components. As shown in FIG. 16F, synthesis is carried out withthe noise and the bright portion BR on the blue back remaining.

Condition setting for chroma key processing will now be described.

FIG. 17 is a flowchart showing the processing procedures of the centralprocessing unit 10 at the time of setting conditions for chroma keyprocessing, which is carried out in the above-described manner. Thecentral processing unit 10 displays a key parameter setting screen and apreview screen on the monitor 9, and accepts conditions for chroma keyprocessing in response to the operation carried out by the operatorthrough these screens. The key parameter setting screen is a screen forcondition setting for chroma key processing, and the preview screen is ascreen for confirmation of chroma key processing using a still picture.The central processing unit 10 carries out events registered on the keyparameter setting screen and the preview screen themselves in responseto the operation by the operator, or carries out events registered onbuttons in these screens in response to the operation by the operator,thereby carrying out processing which will now be described.

That is, when the operator designates an editing target and selectschroma key processing in a predetermined menu screen, the centralprocessing unit 10 shifts from step SP1 to step SP2 and controls theentire operation to display the key parameter setting screen and thepreview screen on the monitor 9.

Then, the central processing unit 10 shifts to step SP3 so as to acceptinput of the center color CO (FIG. 6), and at the subsequent step SP4,sets the representative points P0 to P25 (FIG. 6) with default valueswith reference to the center color CO. Then, the central processing unit10 shifts to step SP5 so as to update the preview screen by using thecenter color CO and the representative points P0 to P25. At thesubsequent step SP6, when the operator operates an operating element ofcompletion of parameter setting, the central processing unit 10 shiftsto step SP7 to end this processing procedure. Thus, the centralprocessing unit 10 subsequently sets the look-up tables 29, 34, ifnecessary.

On the other hand, in the case where the operator who has confirmed thepreview screen operates the key parameter setting screen, since theresult of chroma key processing desired by the operator is not obtained,the central processing unit 10 shifts from step SP6 to step SP8 so as toaccept operation by the operator through the key parameter settingscreen. Thus, the central processing unit 10 accepts updating of thecenter color CO and the representative points P0 to P25 so as to adjustthe parameter of chroma key processing, and then returns to step SP5 toupdate the preview screen.

FIG. 18 is a schematic view showing this preview screen 102.

This preview screen 102 is constituted by a foreground picture(Foreground), a background picture (Background), a key signal picture(Key), and first to third synthetic pictures (Mix1 to Mix3 ). To theforeground picture (Foreground), one frame of still picture selected bythe operator from an editing target material designated by an IN-pointand an OUT-point is allocated. To the background picture (Background),one frame of still picture selected by the operator from an editingmaterial for the background is similarly allocated. On the other hand,the key signal picture (Key) is formed by displaying a key signalgenerated by the set center color CO and representative points P0 toP25. To the first to third synthetic pictures (Mix1 to Mix3), stillpictures formed by arithmetic processing of Equations (15), (16) and(17) using the generated key signal are allocated.

The central processing unit 10 of FIG. 1 loads the correspondingforeground picture and background picture to the frame buffers 3, 4 bythe hard disk drive 2, and carries out processing at the key signalgenerating section 6 and the picture synthesizing section 7 byarithmetic processing, thereby generating and displaying the key signalpictures. The central processing unit 10 forms the preview screen sothat the processing result may be confirmed at an appropriate timing inchanging the setting of the center color CO and the representativepoints P0 to P25. Thus, the easiness in handling is improved. Also, thecentral processing unit 10 forms the preview screen 102 so that theprocessing results of three types of chroma key processing allocated tothe editing device 1 may be compared and confirmed. Thus, the easinessin handling for the operator is improved.

In the preview screen 102, a display showing the name of each picture isarranged at an upper left part of each picture. At an upper right partof each picture, a button (full) B1 for enlarged display of each pictureon the entire display screen and a button (FB) B2 for displaying thepicture on a dedicated monitor unit are arranged. When these buttons B1,B2 are clicked by using the mouse 13, the central processing unit 10carries out events registered on the buttons B1, B2, thereby carryingout enlarged display of each picture and displaying the still picture onthe dedicated monitor.

FIG. 19 shows a specific example of the preview screen as describedabove.

FIG. 20 is a schematic view showing the key parameter setting screen101. This key parameter setting screen 101 is constituted by a centercolor setting section 111, a color space display section 112, and avector scope display section 120 so that the setting of the center colorCO and the representative points P0 to P25 may be changed by operatingeach display section.

FIG. 21 shows a specific example of the key parameter setting screen.

FIG. 22 is a schematic view showing the relation between the centercolor setting section 111 in the key parameter setting screen 101 ofFIG. 20 and chroma key processing. In this center color setting section111, the same still picture as the foreground picture in the previewscreen 102 is displayed as the foreground picture (foreground). Adisplay area ARC for the center color CO and display areas for theluminance level Y and the hues U, V of the center color CO are formedadjacently to the foreground picture.

When the short area ARC1 is designated by operating the mouse 13 on theforeground picture (foreground), the central processing unit 10 of FIG.1 carries out an event registered on this display area of the foregroundpicture. The central processing unit 10 thus reads out picture data ofthis short area ARC1 from the frame buffer 3 and calculates the averagevalue of the luminance level and the color difference level, therebycalculating the luminance level Y0 and the hues U0, V0 of the centercolor CO. In addition, the central processing unit 10 displays thecenter color in the display area ARC for the center color CO by thecalculated luminance level Y0 and hues U0, V0, and displays thecalculated levels in the display areas for the luminance level Y and thehues U, V.

The central processing unit 10 sets the representative points P0 to P25by using default values with reference to the center color CO, and setstwo spherical shapes K1 and K2 to be references for key signalgeneration on a three-dimensional color space. In this case, the centralprocessing unit 10 sets the representative points at predetermineddistances from the center color, respectively.

After thus setting the center color CO, when a predetermined position onthe foreground picture (foreground) is clicked by operating the mouse 13as indicated by x, the central processing unit 10 similarly carries outan event registered on the foreground picture (foreground), therebycarrying out processing procedures shown in FIG. 23. Specifically, thecentral processing unit 10 shifts from step SP11 to step SP12 to obtainthe coordinate clicked on the foreground picture (foreground), and atthe subsequent step SP13, reads out picture data of the clicked positionfrom the frame buffer 3.

Then, the central processing unit 10 calculates the luminance level yand the color difference levels u, v of the picture data, and at thesubsequent step SP 14, carries out arithmetic processing of thefollowing equation. Thus, the central processing unit 10 calculates adistance d from the center color CO(Y0, U0, V0) with respect to thepixel of the clicked position.

d=((y−Y0)²+(u−U0)²+(v−V0)²)^(½)  (18)

Then, the central processing unit 10 shifts to step SP15 and carries outarithmetic processing of the following equation, thereby calculatingangles p and t in the θ-direction and the φ-direction based on thecenter color CO as the origin, as shown in FIG. 24. These angles p and tin the θ-direction and the φ-direction are expressed by setting thecounterclockwise direction as a positive direction with reference to theU-axis. $\begin{matrix}{p = {\arctan \quad \frac{v - {V0}}{u - {U0}}}} & (19) \\{t = {\arcsin \quad \frac{y - {Y0}}{d}}} & (20)\end{matrix}$

By this series of arithmetic processing, the central processing unit 10calculates a vector B from the center color CO to the clicked pixel,using the polar coordinate. Then, the central processing unit 10 shiftsto step SP16. The central processing unit 10 displays the vectors p, t,d of the clicked pixel with reference to the luminance level y, the huesu and v, and the center color CO, adjacently to the display of thecenter color. Then, the central processing unit 10 shifts to step SP17to end this processing procedure.

After thus setting the center color CO, the central processing unit 10forms the key parameter screen so that the relation with the centercolor CO may be easily grasped by clicking a portion to be replaced withthe background at an appropriate timing. Thus, easiness in handling isimproved. In this first embodiment, when the mouse is thus clicked onthe foreground picture, the central processing unit 10 displays a markerat a corresponding position in the vector scope display section and thecolor space display section as later described, thereby improvingeasiness in handling.

In the operation in the vector scope display section as later described,a portion to be operated to enable replacement of the correspondingportion with the background without affecting portions of other colorsmay be determined from the angles θ and φ. Also, the relative relationwith the position of a key edge where the change of key signal isswitched is determined from the distance information d. Thus, the sizeof the outer sphere K1 or the inner sphere K2 may be varied foradjustment of chroma key processing.

In the center color setting section 111 (FIG. 22), an admission button(OK) 3B corresponding to determination processing of step SP6 (FIG. 17)is arranged on the upper side of the display area ARC for the centercolor CO. When this button (OK) is clicked by operating the mouse 13,the central processing unit 10 ends the condition setting processing ofchroma key processing.

On the lower right side, change mode switch buttons (outer) B4 and(inner) B5 with respect to the outer sphere K1 and the inner sphere K2are arranged. The change mode switch buttons B4, B5 are constituted bytoggle switches for switching the on-off control action. When thebuttons B4, B5 are clicked to be on by operating the mouse 13, a reddisplay is formed in the buttons B4, B5.

In the case where the change mode switch button B4 with respect to theouter sphere K1 is on, when the outer sphere K1 or the inner sphere K2is held and drawn by the mouse 13 in the color space display section,the central processing unit 10 changes the coordinate of therepresentative points P0 to P25 with reference to the center point CO soas to uniformly enlarge or reduce the size of the outer sphere K1 inaccordance with the drawing operation. Similarly, in the case where thechange mode switch button B5 with respect to the inner sphere K2 is on,when the outer sphere K1 or the inner sphere K2 is held and drawn by themouse 13 in the color space display section 112, the central processingunit 10 changes the coordinate of the representative points P0 to P25with reference to the center point CO so as to uniformly enlarge orreduce the size of the inner sphere K2 in accordance with the drawingoperation.

Thus, the central processing unit 10 forms the center color settingsection 11 1 so that desired processing results may be obtained by asimple operation, along with the information displays y, u, p, t, d atthe positions clicked by the mouse 13.

In addition, in the center color setting section 111, a chroma keyprocessing selection button B6 is arranged below the change mode switchbutton (inner) B5. This button B6 is constituted by a toggle switch andcan select chroma key processing by the editing device 1 from theprocessing corresponding to the Equations (15) to (17).

In the center color setting section 111, a reset button (rst) B7 isarranged adjacently to these buttons. When the reset button B7 isclicked by the mouse 13, the central processing unit 10 resets thecoordinate of the representative points P0 to P25 to the initial defaultvalues. Thus, the central processing unit 10 controls the entireoperation so as to easily return to the initial state even when theconditions are changed in various manners.

FIG. 25 is a schematic view showing the vector scope display section120. This vector scope display section 120 includes first to fourthdisplay sections 121 to 124, and adjustment buttons allocated to therespective display sections. In the first to third display sections 121to 123, reference planes perpendicular to the Y-axis are set to crossrepresentative points on UY planes arranged at angles of −45 degrees, 0degree, and 45 degrees with respect to the U-axis, and lines formed bythe reference planes crossing the inner sphere K1 and the outer sphereK2 and a projected picture of pixels of a neighboring foreground pictureare displayed and formed. Thus, distribution of the pixels of theforeground picture may be intuitively grasped through the first to thirddisplay sections 121 to 123, with respect to the direction along theY-axis.

Specifically, in the first display section 121, as shown in FIG. 26,reference planes H1L and H2L are set to cross representative points onthe UY plane arranged at an angle of −45 degrees with respect to theU-axis, and lines formed by the reference planes H1L and H2L crossingthe inner sphere K1 and the outer sphere K2 are displayed. Also, apicture formed by projecting pixels of a luminance level not higher thana predetermined luminance level set with reference to the planes H1L andH2L is displayed. At this point, each pixel is set to the predeterminedluminance level corresponding to the reference planes H1L and H2L and isdisplayed in accordance with the hue of each pixel.

On the other hand, in the second display section 122, as shown in FIG.27, a reference plane HC crossing representative points arranged on theU-axis is set, and a line formed by the reference plane HC crossing theinner sphere K1 and the outer sphere K2 is displayed. In addition, apicture formed by projecting pixels of a predetermined luminance levelset with reference to the reference plane HC is displayed. At thispoint, each pixel is set to the predetermined luminance levelcorresponding to the reference plane HC and is displayed in accordancewith the hue of each pixel.

In the third display section 123, as shown in FIG. 28, reference planesH1H and H2H crossing representative points on the UY plane arranged atan angle of 45 degrees with respect to the U-axis are set, and linesformed by the reference planes H1H and H2H crossing the inner sphere K1and the outer sphere K2 are displayed. Also, a picture formed byprojecting pixels of a luminance level not higher than a predeterminedluminance level set with reference to the planes H1H and H2H isdisplayed. At this point, each pixel is set to the predeterminedluminance level corresponding to the reference planes H1H and H2H and isdisplayed in accordance with the hue of each pixel.

The lines crossing the inner sphere K1 and the outer sphere K2 aredisplayed in an octagonal shape so that the corresponding relation withthe representative points may be easily grasped and so that the portionscorresponding to the representative points may be held by the mouse 13to enable easy change of the positions of the representative points.

Below the first to third display sections 121 to 123, a pair of buttonsB7A and B7B for adjusting the distance from the center color CO to therepresentative point, and a scroll bar B8 for adjusting and displayingthe distance are arranged. On an upper part of the scroll bar B8, thedistance from the center color CO to the representative point isdisplayed with a numeral. Thus, by comparison with the distance d fromthe center color CO displayed in the center color setting section, thecentral processing unit 10 may operate these buttons to easily adjustthe characteristics of the chroma key.

Further below the first to third display sections 121 to 123, selectionbuttons B9A and B9B for selecting the inner sphere K1 and the outersphere K2 are arranged, and eight buttons B10A to B10H corresponding tothe representative points of the inner sphere K1 or the outer sphere K2in each display section are arranged below the selection buttons. Thecentral processing unit 10 forms a red display in each of the selectionbuttons B9A and B9B in response to the selection operation of theselection buttons B9A and B9B. Also, the central processing unit 10switches the change of the representative points to an acceptable stateby operating the buttons arranged below the selection buttons B9A andB9B and by operating the buttons B7A, B7B and the scroll bar B8, withrespect to the inner sphere K1 or the outer sphere K2.

Specifically, when any one of the buttons B10A to B10H is operated to beon after the button B9B for the outer sphere K2 is clicked by the mouse13, the central processing unit 10 changes the distance from the centercolor CO to the representative point of the outer sphere K2corresponding to the operated button B10A to B10H, in response to theoperation of the buttons B7A and B7B or in response to the operation ofthe button in the scroll bar B8. Thus, as indicated by a broken line inFIG. 28, the position of the representative point constituting the outersphere K2 is changed to adjust the outer sphere K2.

When a display position corresponding to each representative point isheld and operated by the mouse 13 in each display section, the centralprocessing unit 10 similarly changes the position of the representativepoint and switches the display in each display section in response tothe operation by the mouse 13. Similarly, when the representative pointof the inner sphere K1 or the outer sphere K2 is changed by operation inthe color space display section, the display of the inner sphere K1 andthe outer sphere K2 in each display section is changed.

On the other hand, in the fourth display section 124, a reference planeH3 parallel to the UY plane passing the center color CO is set, and aline formed by the reference plane H3 crossing the inner sphere K1 andthe outer sphere K2 is displayed, as shown in FIG. 29. In addition,pixels of the three-dimensional color space are projected and displayedon the reference plane H3. At this point, each pixels is set to apredetermined luminance level and is displayed with white color.

Below the fourth display section 124, a pair of buttons B7A and B7B foradjusting the distance from the center color CO to the representativepoint, and a scroll bar B8 for adjusting and displaying the distance arearranged. On an upper part of the scroll bar B8, the distance from thecenter color CO to the representative point is displayed with a numeral.Thus, by comparison with the distance d from the center color COdisplayed in the center color setting section, the central processingunit 10 may operate these buttons to easily adjust the characteristicsof the chroma key.

Further below the fourth display section 124, selection buttons B9A andB9B for selecting the inner sphere K1 and the outer sphere K2 arearranged, and two buttons B10A and B10B corresponding to therepresentative points on the Y-axis of the inner sphere K1 or the outersphere K2 are arranged below the selection buttons. The centralprocessing unit 10 forms a red display in each of the selection buttonsB9A and B9B in response to the selection operation of the selectionbuttons B9A and B9B. Also, the central processing unit 10 accepts thechange of the representative points by operating the buttons B10A, B10Barranged below the selection buttons B9A and B9B and by operating thebuttons B7A, B7B and the scroll bar B8, with respect to the inner sphereK1 or the outer sphere K2.

With respect to the fourth display section 124, when a display portioncorresponding to each representative point is held and operated by themouse 13, the central processing unit 10 similarly changes the positionof the representative point and switches the display in the displaysection in response to the operation by the mouse 13. Similarly, whenthe representative point of the inner sphere K1 or the outer sphere K2is changed by operation in the color space display section 112, thedisplay of the inner sphere K1 and the outer sphere K2 in each displaysection is changed.

FIG. 30 is a flowchart showing processing procedures of projection ofthe foreground picture in the central processing unit 10 in thusdisplaying the inner sphere K1 and the outer sphere K2. When the innersphere K1 and the outer sphere K2 are set with default values, or whenthe position of a representative point is changed by operation in thecolor space display section 112 and the vector scope display section120, or when a center color is newly selected in the center colorsetting section, the central processing unit 10 sequentially carries outevents registered to the vector scope display section 120, therebycarrying out the processing procedures.

Specifically, the central processing unit 10 shifts fro step SP21 tostep SP22 so as to set lightness (luminance level) segments. As shown inFIG. 31, the central processing unit 10 sets threshold values LYL andLYH of luminance level with reference to the luminance levels Y1L, Y2L,YV, Y1H, Y2H of the reference planes H1L, H2L, HV, H1H, H2H set in thefirst to third display sections 121 to 123, and thus sets threeluminance level segments lo, mid, high corresponding to the first tothird display sections 121 to 123.

Then, the central processing unit 10 shifts to step SP23 so setreference values YH, YC, YL of lightness (luminance level) with respectto the respective segments high, mid, lo. In this embodiment, thereference value YL corresponding to the first display section 121 is setto be the average value of the luminance levels Y1L, Y2L of thereference planes H1L, H2L set with respect to the first display section121. The reference value YH corresponding to the third display section123 is set to be the average value of the luminance levels Y1H, Y2H ofthe reference planes H1H, H2H set with respect to the third displaysection 123. The threshold values LYL, LYH are set to be the averagevalue of the reference values YL, YC and the average value of thereference values YC, YH, respectively.

Subsequently, the central processing unit 10 shifts to step SP24 tocolor the display in the first to third display sections with black,thereby initializing the first to third display sections 121 to 123.Then, the central processing unit 10 shifts to step SP25. The centralprocessing unit 10 loads picture data from the frame buffer 3 withrespect to one pixel of the foreground picture.

At the subsequent step SP26, the central processing unit 10 determinesthe segment high, mid, or lo to which the loaded picture data belongwith reference to the luminance level of the picture data. In addition,the central processing unit 10 distributes the picture data to any oneof the segments high, mid, lo from the result of determination, and setsthe luminance level of the picture data to the reference value of thecorresponding segment.

Then, the central processing unit 10 shifts to step SP27 to display thepicture data in the display section corresponding to the segment towhich the picture data are distributed. At this point, the centralprocessing unit 10 sets the horizontal direction and the verticaldirection of each display section to the U-axis and the V-axis,respectively. The central processing unit 10 selects the positioncorresponding to the hue of the picture data and carries out pointdisplay of the picture data at this position. In addition, the centralprocessing unit 10 carries out point display by using the luminancelevel set to the reference value at step SP23 or by using the hue of thepicture data.

Thus, on completion of point display with respect to one pixel, thecentral processing unit 10 shifts to step SP28 to determine whether ornot point display is completed with respect to all the pixels of theforeground picture. If a negative result is obtained, the centralprocessing unit 10 returns to step SP25. The central processing unit 10repeats the processing procedures of steps SP25, SP26, SP27, SP28 andSP25 so as to display each pixel in the corresponding display section inaccordance with the luminance level of each pixel forming the foregroundpicture. On completion of display of all the pixels, the centralprocessing unit 10 shifts to step SP29.

The central processing unit 10 obtains information related to a keyrange. The information related to a key range is coordinate data oflines formed by the above-described reference planes H1L, H2L, HC, H1H,H2H crossing the inner sphere K1 and the outer sphere K2. On obtainingthis information, the central processing unit 10 shifts to step SP30 todisplay the lines in each display section.

Subsequently, the central processing unit 10 shifts to step SP31 todetermine whether the key range (that is, the position of therepresentative points) is changed or not. If a positive result isobtained, the central processing unit 10 returns to step SP22. Thus,when the representative points are changed, the central processing unit10 newly switches the display of the display section so as to enableeasy determination of distribution of each pixel with respect to theinner sphere K1 and the outer sphere K2 formed by the representativepoints.

On the contrary, if a negative result is obtained at step SP31, thecentral processing unit 10 shifts from step SP31 to step SP32 todetermine whether the center color CO is changed or not. If a positiveresult is obtained, the central processing unit 10 returns to step SP22to newly switch the display of each display section. Thus, even when thecenter color CO is changed, the central processing unit 10 enables easydetermination of distribution of each pixel with respect to the innersphere K1 and the outer sphere K2 changed in accordance with this changeof the center color.

On the contrary, if a negative result obtained at step SP32, the centralprocessing unit 10 shifts from step SP32 to step SP33 to end thisprocessing procedure.

When the foreground picture is clicked by the mouse 13 in the centercolor setting section 111, the central processing unit 10 displays amarker at a corresponding position in the first to fourth displaysections 121 to 124, along with the display of the angles p, t in thecenter color setting section 111. Thus, in the editing device 1, therelation between each pixel of the foreground picture and the innersphere K1 and the outer sphere K2 may be easily grasped so that easinessin handling may be improved.

FIG. 32 is a schematic view showing the color space display section 112.This color space display section 112 includes a picture display section115 for displaying a picture formed by viewing the inner sphere and theouter sphere located in the three-dimensional color space and the pixelsof the foreground picture from a predetermined direction, and buttonsfor changing the contents and viewpoint of the picture.

Specifically, at an upper left part of the color space display section112, a button (uv) for designating the UV plane, a button (yu) fordesignating the YU plane, and a button (yv) for designating the YV planeare arranged. As shown in FIGS. 33A-33D, the central processing unit 10displays a picture formed by viewing the color space from a viewpointset in a predetermined direction, onto the picture display section 115(FIG. 33(A)). When the respective buttons (uv), (yu), (yv) are operated,the central processing unit 10 carries out events registered to thebuttons so as to display pictures formed by viewing the UV plane, the YUplane, and YV plane (FIGS. 33(B) to (D)).

In the color space display section 112, a rotation operation button(rot) is arranged adjacently to the button (uv) for designating the UVplane. As shown in FIGS. 34A-34D, when this button (rot) is operated tobe on in the state where the picture formed by viewing the color spacefrom the predetermined direction is displayed on the picture displaysection 115 (FIG. 34(A)), the central processing unit 10 sets theY-axis, U-axis, and V-axis designated by the subsequent operation of themouse 13 to the rotational center axis in this state of display. Inaddition, the central processing unit 10 rotates the color space asindicated by an arrow with respect to the rotational center axis, inresponse to the operation of the mouse 13 (FIGS. 34(B) to (D)). Thus,the-central processing unit 10 forms the color space display section 112so that the relation between the inner sphere K1, the outer sphere K2and each pixel of the foreground picture may be easily grasped byviewing the color space from various directions, if necessary.

In the color space display section 112, a button (in) for designatingthe inner sphere K1 and a button (out) for designating the outer sphereK2 are also arranged. When the button (in) or (out) is operated to be onby the mouse 13, the central processing unit 10 displays the innersphere K1 or the outer sphere K2 on the picture display section 115.Also, in the color space display section 112, a button (mix) forswitching the display mode of the inner sphere K1 and the outer sphereK2 is arranged. When this button (mix) is operated by the mouse 13, thecentral processing unit 10 switches the display of the inner sphere K1and the outer sphere K2 between display on an opaque plane and displayon a semi-transparent plane. Thus, the central processing unit 10 formsthe color space display section 112 so that the relation between eachpixel of the foreground picture and the inner sphere and the outersphere may be easily confirmed by switching the display of the innersphere K1 and the outer sphere K2, if necessary, and also by switchingthe display mode thereof.

In addition, in the color space display section 112, a button (wire) forswitching the display form of the inner sphere K1 and the outer sphereK2 is arranged. When this button (wire) is operated to be off, thecentral processing unit 10 displays the inner sphere K1 and the outersphere K2 in the form of plane. When the button (wire) is operated to beon, the central processing unit 10 displays the inner sphere K1 and theouter sphere K2 in the form of wire frame defined by lines connectingadjacent representative points. Thus, the central processing unit 10forms the color space display section 112 so that the relation betweeneach pixel of the foreground picture and the inner sphere and the outersphere may be easily confirmed by switching the display form of theinner sphere K1 and the outer sphere K2, if necessary.

Moreover, in the color space display section 112, a button (mov) forsequentially changing the display of the color space by a small angle isarranged. When this button (mov) is operated, the central processingunit 10 sets a virtual rotational axis corresponding to the verticaldirection of the picture display section 115 into the color space, anddisplays the color space while sequentially displacing the color spaceby a predetermined angle around the rotational axis. Thus, the centralprocessing unit 10 forms the color space display section 112 so that therelation between each pixel of the foreground picture and the innersphere and the outer sphere may be easily confirmed by sequentiallychanging the viewpoint.

In the color space display section 112, a button (line) for switchingthe display form of each pixel of the foreground picture is arranged.When this button (line) is operated to be off, the central processingunit 10 maps each pixel of the foreground picture into the color spaceby using a point image. On the contrary, when this button (line) isoperated to be of, the central processing unit 10 connects the pointimages of the respective pixels on the foreground screen by a brightline in the order of raster operation. Thus, the central processing unit10 forms the color space display section 112 so that the relationbetween each pixel of the foreground picture and the inner sphere andthe outer sphere may be easily confirmed by switching the display formof each pixel, if necessary.

In the color space display section 112, a button (prv) for switchingdisplay/non-display with respect to the pixels of the foreground pictureis arranged adjacently to the button (yu) for designating the YU plane.When this button (prv) is operated to be off, the central processingunit 10 suspends display of the pixels of the foreground picture. Thisdisplay of the pixels uses the hue and luminance level corresponding toeach pixel so that the relation between the inner sphere and the likeand each pixel may be easily grasped even when the color space is viewedfrom various viewpoints.

Also, in the color space display section 112, a button (cur) forswitching display/non-display of the UV axis is arranged. The centralprocessing unit 10 switches the display of the U-axis and V-axis inresponse to the on-off control section of the button (cur). In addition,in the color space display section 112, a button (ring) for switchingdisplay/non-display with respect to a hue ring R is arranged. Thecentral processing unit 10 arranges a hue ring R which indicates the hueat each position by a color, on the UV plane, and switches the displayof this ring R in response to the on-off control action of this button(ring). Thus, the central processing unit 10 forms the color spacedisplay section 112 so that the relation between the inner sphere, theouter sphere and the foreground picture on the three-dimensional colorspace may be visually grasped even when the color space is viewed fromvarious viewpoints.

In the color space display section 112, a button (col) for switchingdisplay/non-display of a marker is arranged. When this button (col) isoperated to be on, the central processing unit 10 displays a box-shapedmarker M at a portion of each color expressed by a color bar. Thus, thecentral processing unit 10 forms the color space display section 112 sothat the relation between the inner sphere, the outer sphere and theforeground picture on the three-dimensional color space may be easilygrasped even by an operator who is accustomed to the use of ordinaryimage equipments.

In the color space display section 112, a button (Y) for switchingdisplay/non-display of the Y-axis is also arranged. The centralprocessing unit 10 switches the display of the Y-axis in response to theon-off control action of this button (Y). Also, in the color spacedisplay section, a button (scope) for enlarging/contracting the displayof the picture display section 115 is arranged. When this button (scope)is operated, the central processing unit 10 enlarges or contracts thedisplay of the color space in response to the subsequent operation ofthe mouse 13.

In the color space display section 112, a button (rud) for switching toa display mode in consideration of the number of pixels with respect tothe display of the pixels of the foreground picture is arranged. Whenthis button (rud) is operated to be off, the central processing unit 10forms a point image of each pixel in the color space in accordance withthe luminance level and hue of each pixel. In this case, the centralprocessing unit 10 forms point images overlapping at the same portion,with respect to a plurality of pixels of the same hue and the sameluminance level. On the contrary, when this button (rud) is operated tobe on, the central processing unit 10 corrects the luminance level andhue of each pixel by using predetermined random numbers and displays thecorrected luminance level and hue. Thus, the central processing unit 10forms the color space display section 112 so that, with respect to aplurality of pixels of the same hue and the same luminance level,concentration of the number of pixels may be visually sensed easily byslightly shifting the display positions of point images and displayingthe point images to have an area corresponding to the number of pixels.

For such display, scroll bars for varying the degree of display arearranged on the left side of the color space display section 112. Whenthe lowermost scroll bar (a) of these scroll bars is operated, thecentral processing unit 10 varies the overall brightness in the picturedisplay section 115. When a scroll bar (rud) is operated, the centralprocessing unit 10 varies the maximum value of the random numbers usedin the operation of the button (rud). When a scroll bar (hnd1) isoperated, the central processing unit 10 enlarges/contracts the displayof the representative points located on the inner sphere K1 and theouter sphere K2 and the display of the center color.

In the picture display section 115, when the display of therepresentative points and the display of the center color which arevaried in size by thus operating the scroll bar (hnd1) are held andmoved by the mouse 13 (FIG. 4), the central processing unit 10 changesthe representative points and the center color in accordance with thismovement by the mouse. At this point, when the change mode switchbuttons (outer) B4 and (inner) B5 are operated to be on, the radius ofeach representative point of the inner sphere K1 or the outer sphere K2is varied so as to correspond to the change in radius due to the changein any one representative point. Thus, the overall size of the innersphere K1 or the outer sphere K2 is varied. When the center color ischanged, each representative point is changed so that the relationbetween the center color and each representative point is maintained.

When the representative points are changed in the vector scope displaysection, the central processing unit 10 changes the display of the innersphere K1 and the outer sphere K2 in the color space display section 112in accordance with this change in the vector scope display section. Whenthe center color is changed by the center color setting section 111, thecentral processing unit 10 switches the display of the inner sphere K1and the outer sphere K2, similarly to the case where the center color ischanged in the picture display section 115. In addition, when any one ofthe pixels is selected by clicking the mouse 13 in the center colorsetting section 111, the central processing unit 10 displays a marker inthe picture display section 115 in accordance with the display of theradius d or the like.

Referring to FIG. 1 again, the operation of the first embodiment of thepresent invention will now be described.

In the editing device 1 of FIG. 1, after image signals SV1, SV2 asediting processing targets are recorded onto the hard disk drive 2, anediting list is prepared by these image signals SV1, SV2. In accordancewith the editing list, the image signals SV1, SV2 are edited andrecorded onto the hard disk drive 2.

In preparation of such editing list, in the editing device 1, when theoperator selects chroma key processing, the key parameter setting screenand the preview screen are displayed on the monitor 9 (as shown in FIGS.25 and 18), and corresponding picture data are inputted to the framebuffers 3 and 4 from the hard disk drive 2. Thus, a foreground pictureand a background picture selected by the operator are displayed in theform of still picture on the key parameter setting screen and thepreview screen.

In the editing device 1, when the operator designates a part of an areaallocated to the background picture as a short shape by operating themouse 13 in the foreground picture on the key parameter setting screen(FIG. 25), picture data corresponding to this area are read out from theframe buffer 3 and averaged so as to determine the center color CO (Y0,U0, V0) of chroma key processing. By this averaging processing, theediting device 1 effectively avoids the influence of noise and the like,and sets a color desired by the operator to the center color by a simpleoperation.

On the YUV space, two types of representative points spaced from thecenter color CO by a predetermined distance are set at a conformalspacing of 45 degrees centering the center color CO. By these two typesof representative points, double spherical shapes K1, K2 centering thecenter color CO are set with default values. Thus, the editing device 1sets boundaries K1 and K2 switching the characteristics of the keysignal KEY, under standard conditions.

Having thus setting the center color CO and the representative points,the editing device 1 displays, on the preview screen, a picture of achroma key signal under the standard conditions and three types ofprocessing results of chroma key processing using this chroma keysignal. Thus, the processing results under the standard conditions maybe confirmed. Also, the position of the inner sphere K1 and the outersphere K2 under the standard conditions on the color space and eachpixel of the foreground picture are displayed in the color space displaysection (FIG. 32). In addition, a picture formed by projecting linescrossing the inner sphere K1 and the outer sphere K2 with respect to aplane on the color space set with reference to each representative pointand each pixel of the foreground picture is displayed in the vectorscope display section.

Thus, the characteristics of the key signal are changed in variousmanners through the key signal characteristic setting screen formed bythe vector scope display section, the color space display section andthe center color setting section, and the result of this processing maybe confirmed on the preview screen. Even when each pixel of theforeground picture is expressed in the three-dimensional color space soas to generate the key signal, the result of processing may be graspedin an interactive mode or intuitively, so that desired processingresults may be obtained.

Specifically, in the display in this vector scope display section, theediting device 1 projects and displays the lines crossing the innersphere K1 and the outer sphere K2 with respect to three types ofreference planes perpendicular to the Y-axis and the correspondingpixels of the foreground picture. Thus, the operator who operates theediting device 1 can visually grasp distribution of colors of theforeground picture, and can carry out desired chroma key processing by asimple operation, even when each pixel of the foreground picture isexpressed on the three-dimensional color space so as to carry out chromakey processing.

Also, since the boundaries K1 and K2 for switching the characteristicsof the key signal KEY are displayed together, a range which is to bereplaced with the background by chroma key processing (that is, a rangeconstituted by pixels distributed substantially on the inner side of theinner sphere K1) and a range which is not to be replaced with thebackground by chroma key processing (that is, a range constituted bypixels distributed substantially on the outer side of the outer sphereK2) may be visually confirmed. Therefore, even when each pixel of theforeground picture is expressed on the three-dimensional color space soas to carry out chroma key processing, the operator can set theconditions while presuming the processing results. Thus, easiness inhandling is improved.

In the state of such setting, if the operator confirms the previewscreen to find that a desired color is not replaced with the backgroundin the presumed manner, the operator shifts the display of lines on theinner sphere K1 and the outer sphere K2 in the vector scope displaysection by operating the mouse 13, thereby changing the representativepoints and changing the conditions for chroma key processing (FIG. 28).

Also, the buttons B7A, B7B, B10A to B10H and the scroll bar B8 arrangedbelow the first to fourth display sections are operated, therebychanging the representative points and changing the conditions forchroma key processing (FIGS. 26 to 29).

In this case, with respect to a color desired by the operator, eachpixel is displayed on the basis of the hue and reference lightness ofeach pixel in the first to third display sections in the vector scopedisplay section, and in the form of black and white display in thefourth display section concerning the luminance level. Therefore, theoperator may change the conditions for chroma key processing by easilyand securely determining a portion where the display of the line on theinner sphere K1 and the outer sphere K2 should be held and moved by themouse 13.

When the foreground picture is clicked by the operator 13 in the centercolor setting section, the angle and direction of the clicked pixelviewed from the center color are displayed, and the relation with anarea to be replaced with the background picture near the center color isdisplayed by a numerical value. This relation is displayed to correspondto the display in the vector scope display section, by positioninformation p, t, d in the form of polar coordinate corresponding thesetting of the representative points (FIG. 22). Thus, in the vectorscope display section, the conditions for chroma key processing may bechanged by easily and securely determining a portion on the inner sphereK1 and the outer sphere K2 that should be held and moved by the mouse13.

Since the conditions for chroma key processing may be changed simply byholding and moving the display of the line on the inner sphere K1 andthe outer sphere K2 by the mouse 13, easiness in handling is improved.

Also, the buttons B7A, B7B, B10A to B10H and the scroll bar B8 arearranged in the form corresponding to the display of the positioninformation, below the first to third display sections. As the settingof representative points including key signal setting information isaccepted by using the buttons B7A, B7B, B10A to B10H and the scroll barB8, the conditions for chroma key processing may be changed easily andsecurely by operating any one of the buttons B7A, B7B, B10A to B10H andthe scroll bar B8.

In these cases, in the color space display section, a picture, formed byviewing from a desired direction a three-dimensional color space onwhich each pixel of the foreground picture, the inner sphere K1 and theouter sphere K2 are arranged, is displayed, and the viewpoint is changedin various manners by operating the mouse 13 so as to change thisdisplay. Thus, the relation between the inner sphere K1, the outersphere K2 and each pixel of the foreground picture may be graspedthrough the color space display section, if necessary, and theconditions for chroma key processing may be changed by confirming thecolor space display section and operating the vector scope displaysection.

At this point, in the color space display section, the display of eachpixel, the inner sphere K1 and the outer sphere K2 is switched inresponse to the operation of the button, and the ring R indicating thehue and the YUV axes are displayed. In addition, a marker is displayedcorresponding to the designation of a pixel in the foreground picture inthe center color setting section. Thus, the relation between the innersphere K1, the outer sphere K2 and each pixel of the foreground picturemay be visually grasped easily and accurately, thus simplifyingadjustment of chroma key processing.

In the color space display section, too, when the display of therepresentative points arranged on the surface of the inner sphere K1 andthe outer sphere K2 and the display of the center color are held andmoved by the mouse 13, the representative points and the center colorare changed in accordance with this movement. Thus, the conditions forchroma key processing are changed.

When the representative points and the like are thus set and desiredchroma key processing results are confirmed on the preview screen, theediting device 1 carries out subsequent processing by the operation ofselection of chroma key processing in the center color setting sectionand completion of condition setting.

Specifically, in the editing device 1, the latitude φ and the longitudeθ are sequentially displaced by arithmetic processing by the centralprocessing unit 10, and the distances ri and ro from the origin CO tothe boundaries K1 and K2 on the YUV space are calculated (FIG. 9). Atthis point, in the editing device, the distances ri and ro arecalculated by interpolation processing using the latitude φ and thelongitude θ as parameters (Equations (1) and (2)). In addition, from theresult of this calculation, the clip point CLIP and the gain GAINprescribing the characteristic of the chroma key signal KEY arecalculated (FIGS. 11A and 11B). In the editing device 1, the calculatedclip point CLIP and gain GAIN are outputted to the key signal generatingsection 6, and thus, the look-up table 29 (FIG. 2) using the latitude φand the longitude θ as addresses is formed by simple arithmeticprocessing.

In the editing device 1, if the operator selects color cancelprocessing, the clip point CLIPC and the gain GAINC prescribing thecharacteristic of the color cancel key signal are similarly calculatedevery longitude θ from the double boundaries set for color cancel. Bythese calculated clip point CLIPC and gain GAINC, the look-up table 34for color cancel processing is formed in the key signal generatingsection 6.

After the loop-up tables 29, 34 are thus formed, if the operatordesignates preview and editing start, in the editing device 1, the imagesignal SV 1 of the foreground picture and the image signal SV2 of thebackground picture are inputted to the chroma key processing section 5(FIG. 1) by the hard disk drive 2. With respect tot the image signal SV1of the foreground picture of these image signals, the color-differencesignals U1, V1 are over-sampled by the over-sampling circuits 26U, 26V(FIG. 2) of the key signal generating section 6 and thus transformed todigital signals by the same sampling frequency as for the luminancesignal.

With respect to the image signal SV1 thus processed, the distance rmfrom the origin CO and the angle θ from the reference axis are detectedon the UV coordinate plane with the center color CO as the origin, bythe coordinate transformation circuit 27 of the chroma key signalgenerating section 25. Thus, with respect to the image signal SV1, thelength rm of the line formed by projecting onto the UV plane the line Lconnecting the position of each pixel and the origin CO on the YUV spacewith the center color CO as the origin, and the longitude θ of the lineL on the YUV space are detected.

In addition, with respect to the image signal SV1, the latitude φ andthe distance rM to the origin CO, of the position of each pixel on theYUV space with the center color CO as the origin, are calculated by thecoordinate transformation circuit 28, with reference to the distance rmdetected by the coordinate transformation circuit 27 and the luminancelevel Y.

Thus, with respect to the image signal SV1, the look-up table 29 isaccessed by using the latitude φ and the longitude θ as addresses, andthe clip point CLIP and the gain GAIN corresponding to the latitude φand the longitude θ are set to the key process circuit 30. Thus, withrespect to the image signal SV1, the key signal MXK within a range ofvalues 0 to 1 in accordance with the distance rM from the center colorCO is generated on the basis of the clip point CLIP and the gain GAIN.

Also, with respect to the image signal SV1, the color-difference signalsU1 and V1 outputted from the over-sampling circuits 26U and 26V areinputted to the coordinate transformation circuit 33 of the color cancelkey signal generating section 32, where the color of each pixel isdetected from the angle θC and the distance rC on the UV plane with thecolor cancel reference color COC (U0C, V0C) as the origin. In addition,with respect to the image signal SV1, the look-up table 34 is accessedby using the angle θC as an address, the clip point CLIPC and the gainGAINC corresponding to this angle θC are set to the key process circuit35. Thus, with respect to the image signal SV1, the key signal AMKwithin a range of values 0 to 1 in accordance with the distance rC fromthe color cancel reference color COC is generated on the basis of theclip point CLIPC and the gain GAINC.

Thus, with respect to the image signal SV1, the key signal MXK iscalculated with reference to the distance rm and the latitude θ on theUV space having the center color CO set as the origin, and the keysignal MXK is generated by simple arithmetic processing. By accessingthe look-up table 34 preset with reference to the angle θC as theaddress so as to set the clip point CLIPC and the gain GAINC, the keysignal AMK is generated in a short processing time. In addition, as theclip point CLIPC and the gain GAINC prescribe the characteristic of thekey signal AMK, the key signal AIMK may be generated without carryingout division processing, so that the processing time is reducedaccordingly.

If the operator does not select color cancel processing, in the editingdevice 1, the key signal CCK of the value 1 is outputted from the colorcancel key signal generating section 32. Thus, the image signal SV1 isinputted to the subsequent keyers 21Y, 21U, 21V without being processedby the color cancel circuits 20Y, 20U, 20V (FIG. 12). The image signalSV1 is weighted by the key signal KEY (MXK) outputted from the chromakey signal generating section 25, and is then added by the subsequentmixers 23Y, 23U, 23V to the image signal SV2 outputted from the keyers22Y, 22U, 22V. At this point, since the image signal SV2 is weighted bythe key signal 1−KEY (1−MXK) at the keyers 22Y, 22U, 22V, the imagesignal SV3 processed by chroma key processing as expressed by Equation(15) is outputted from the mixers 23Y, 23U, 23V.

On the other hand, if the operator selects the first color cancelprocessing, in the editing device 1, the key signal CCK (1−AMK) isoutputted from the color cancel key signal generating section 32 withrespect to the color-difference signals. Thus, with respect to the imagesignal SV1, the color-difference signals U1, V1 are weighted by the keysignal CCK at the color cancel circuits 20Y, 20U, 20V, and are theninputted to the subsequent keyers 21Y, 21U, 21V. The image signal SV1 isweighted by the key signal KEY (MXK) outputted from the chroma keysignal generating section 25, and is added by the subsequent mixers 23Y,23U, 23V to the image signal SV2 weighted by the key signal 1−KEY(1−MXK). Thus, the image signal SV3 processed by chroma key processingas expressed by Equation (16) is outputted from the mixers 23Y, 23U,23V.

If the operator selects the second cancel processing, the image signalSV1 is weighted by the key signal CCK (1−AMK) at the color cancelcircuits 20Y, 20U, 20V, and is then weighted by the key signal KEY (AMK)at the subsequent keyers 21Y, 21U, 21V. The image signal SV1 is thenadded by the subsequent mixers 23Y, 23U, 23V to the image signal SV2weighted by the key signal 1−KEY (1−AMK). Thus, the image signal SV3processed by chroma key processing as expressed by Equation (17) isoutputted from the mixers 23Y, 23U, 23V.

In this manner, with respect to the image signal SV3 formed bysynthesizing the foreground picture and the background picture, the keysignal MXK is generated with reference to the luminance signal level andthe color-difference signal level of the image signal SV1, for example,the key signal MXK is generated by discriminating light blue and darkblue. Thus, a synthetic picture of high definition having no incongruitymay be provided.

The effect of the above-described first embodiment of the presentinvention will now be described.

According to the key signal generating device having the above-describedstructure, the position of each pixel of the foreground picture isexpressed in the form of polar coordinate in the three-dimensional YUVcolor space having the center color of a color to be extracted from theforeground picture as the origin, and the key signal KEY (MXK) isgenerated in accordance with the distance RM from the origin in thispolar coordinate. Thus, with a simple structure, the key signal KEY(MXK) may be generated also with reference to the luminance of theforeground picture. For example, the key signal MXK may be generated bydiscriminating light blue and dark blue, and a synthetic picture of highdefinition having no incongruity may be provided.

In addition, the boundaries K1 and K2 are set to surround the origin COso as to generate the key signal KEY in accordance with the position ofeach pixel with respect to the boundaries K1 and K2. Thus, the settingof the chroma key characteristic may be freely changed by changing theboundaries K1 and K2 in various manners, and easiness in handling may beimproved accordingly.

Also, after the chroma key characteristic at each latitude and longitudeis detected by using the look-up table, the key signal KEY is generatedon the basis of the detected characteristic. Thus, the key signal may begenerated simply and in a short time.

In addition, since the characteristic is set by using the clip point andthe gain, arithmetic processing required for generating the key signalKEY may be simplified accordingly.

Also, after the distance rm and angle θ of a line formed by projectingthe line connecting the center color CO and each pixel onto the UV planeare detected, the latitude φ, the longitude θ and the distance rM aredetected. Thus, the position of each pixel on the YUV color space may beexpressed by polar coordinate by simple arithmetic processing, and thetime required for generating the key signal may be reduced accordingly.

Moreover, each of the boundaries K1 and K2 is expressed by 26 points,and the coordinate values of these groups of points are interpolated toform the look-up table. Thus, the conditions for chroma key processingmay be changed in various ways by deforming the boundaries K1 and K2 invarious manners using simple arithmetic processing, and the look-uptable may be formed by simple arithmetic processing.

Also, since the three-dimensional color space is formed with referenceto the luminance signal level and the color-difference signal level, animage signal of a digital signal format (for example, so-called D1format) processed by this type of editing device may be easily processedto generate the key signal.

In addition to these structures, the color cancel key signal isgenerated by similar processing. Thus, a synthetic picture of highdefinition may be generated simply in a short time.

According to the editing device having the above-described structure,since the result of processing by the key signal and the key signalcharacteristic setting screen are displayed, the processing result maybe confirmed in real time and the characteristic of the key signal maybe changed in the interactive mode. Also, the processing result may beintuitively grasped and detailed adjustment may be carried out, ifnecessary. Even when each pixel of the foreground picture is expressedand processed by chroma key processing on the three-dimensional colorspace, desired processing may be carried out by simple operation.

At this point, since the processing result is displayed together withthe foreground picture, the background picture and the key signalpicture, the contents of processing may be visually grasped bycomparison between these pictures, and the adjustment precision may beimproved accordingly.

Also, the characteristic setting screen is constituted by the colorspace display section in which the three-dimensional color space isviewed from a desired viewpoint, and the vector scope display section inwhich the three-dimensional space is projected onto a predeterminedreference plane. Thus, the characteristic of the key signal may bechanged through any one of the screens, if necessary. Even when eachpixel is expressed and processed by chroma key processing on thethree-dimensional color space, easiness in handling may be improved.

In addition to these features, since the characteristic of the keysignal may be changed by changing the representative points using anoperating element including the scroll bar, the processing target pixelmay be grasped by a numerical value so as change the characteristic ofthe key signal.

A second embodiment of the present invention will now be described.

The editing device of the second embodiment has the same structure asthat of the first embodiment, except for processing procedures in thecentral processing unit. In this embodiment, an automatic setting buttonfor chroma key processing conditions is arranged in the center colorsetting section of the key parameter setting screen of the firstembodiment. The automatic setting button is arranged separately forrepresentative points of the inner sphere K1 and for representativepoints of the outer sphere K2.

The central processing unit carries out the same processing proceduresas those in the central processing unit 10 of the editing device 1 ofthe first embodiment, except for processing related to the automaticsetting button. When the foreground picture is clicked by the mouse 13after the automatic setting button for the representative points of theinner sphere K1 or the outer sphere K2 is operated by mouse 13, thecentral processing unit carries out processing procedures shown in FIG.35, thereby changing the position of the representative point so as toreplace each clicked pixel with the background picture or so as not toreplace each clicked pixel with the background picture.

Specifically, the central processing unit shifts from step SP40 to stepSP41 so as to obtain the clicked coordinate and read out picture data atthe clicked position from the frame buffer 3. Then, the centralprocessing unit detects the luminance level y and the color-differencelevels u, v of the picture data, and carries out arithmetic processingof Equations (18) to (20) using the results of this detection so as tocalculate the distance d from the center color CO (Y0, U0, V0) and theangles p and t in the θ-direction and the φ-direction based on thecenter color CO as the origin.

Then, the central processing unit shifts to step SP42 so as to determinewhether or not a representative point exists within a predeterminedangular range, from the detected angles p and t. If a representativepoint having angles θ and φ proximate to the angles p and t exists, thecentral processing unit shifts to step SP43 so as to change thecoordinate value of this neighboring representative point. Specifically,if the operator operates the automatic setting button for therepresentative point of the inner sphere K1, the central processing unitchanges the distance of the corresponding representative point so thatthe distance from the center color CO (Y0, U0, V0) is increased by apredetermined value with respect to the distance d from the center colorCO (Y0, U0, V0) calculated at step SP41. If the operator operates theautomatic setting button for the representative point of the innersphere K2, the central processing unit changes the distance of thecorresponding representative point so that the distance from the centercolor CO (Y0, U0, V0) is decreased by a predetermined value with respectto the distance d from the center color CO (Y0, U0, V0) calculated atstep SP41.

Having thus changed the representative point, the central processingunit shifts to step SP44 so as to determine whether or not theforeground picture is clicked again. If a positive result is obtained,the central processing unit returns to step SP41. If a negative resultis obtained at step SP42, the central processing unit shifts to stepSP46 so as to change the coordinate of neighboring four representativepoints having angles θ and φ proximate to the angles p and t so that thedistance r (φ, θ) from the center color CO (Y0, U0, V0) expressed byarithmetic processing of the following equation corresponds to thecondition set at step SP43. $\begin{matrix}{{r\left( {\varphi,\theta} \right)} = \frac{S}{\left( {{\theta 1} + {\theta \quad 2}} \right) \cdot \left( {{\varphi 1} + {\varphi \quad 2}} \right)}} & (21)\end{matrix}$

S=(roa+x)·θ2·φ2+(rob+x)·θ1·φ2+(roc+x)·θ2·φ1+(rod+x)·θ1·φ1

In this case, if the operator operates the automatic setting button forthe representative point of the inner sphere K1, the central processingunit sets a value x so that the distance from the center color CO (Y0,U0, V0) is increased with respect to the distance d from the centercolor CO (Y0, U0, V0) calculated at step SP41, and changes the distanceof these neighboring representative points. On the other hand, if theoperator operates the automatic setting button for the representativepoint of the inner sphere K2, the central processing unit sets a value xso that the distance from the center color CO (Y0, U0, V0) is decreasedwith respect to the distance d from the center color CO (Y0, U0, V0)calculated at step SP4 1, and changes the distance of these neighboringrepresentative points.

After changing the neighboring four representative points by the equaldistance x, the central processing unit shifts to step SP44. The centralprocessing unit sequentially changes the coordinate of therepresentative points in response to the operation by the operator. Oncompletion of the operation by the operator, the central processing unitshifts from step SP44 to step SP47 to end the processing procedures.

The effect of the above-described second embodiment will now bedescribed.

As shown in FIG. 35, even when the characteristic of the key signal isautomatically set by changing the representative point in response tothe operation by the user, since the processing result and thecharacteristic setting screen are displayed, the processing result maybe confirmed in real time so as to enable detailed adjustment, ifnecessary. Thus, even when each pixel of the foreground picture isexpressed and processed by chroma key processing on thethree-dimensional color space, desired processing may be carried out bysimple operation.

Other embodiments will now be described.

Although in the above-described embodiment, the boundary for chroma keyprocessing is expressed by 26 point, this invention is not limited tosuch embodiment. For example, in the case where points are set at aconformal spacing of 30 degrees or in the case where points are set at anon-conformal spacing, the boundary may be expressed by various numbersof points, if necessary.

Also, though in the above-described embodiment, the boundary for chromakey processing is set by using double substantially spherical shapes,this invention is not limited to such embodiment. For example, thisinvention may also be applied to the case where triple boundaries areset so that the chroma key processing characteristic is set further indetail with reference to the intermediate boundary.

Also, in the above-described embodiment, the latitude and longitude arechanged to calculate the distance to the boundary in advance, and thelook-up table is formed from the result of this calculation so as togenerate the key signal. However, this invention is not limited to suchembodiment. The distance to the boundary may be sequentially calculatedcorresponding to each pixel of the foreground picture so as to generatethe key signal. Also, since it often occurs that the same luminancesignal level and color-difference signal level continue between adjacentpixels in the image signals, the latitude and longitude once calculatedmay be utilized in sequentially calculating the distance to theboundary.

In addition, in the above-described embodiment, the look-up table isaccessed with reference to the latitude and longitude, therebysequentially setting the key signal level in accordance with thedistance from the center color with respect to all the pixels of theforeground picture. However, this invention is not limited to suchembodiment. This invention may also be applied to the case where the keysignal level is detected by accessing the look-up table or by arithmeticprocessing only with respect to part of pixels sampled from theforeground picture so that the key signal level for the other pixels isset from the detected key signal level by using interpolationprocessing.

Also, in the above-described embodiment, each pixel of the foregroundpicture is expressed by the YUV color space having the center color of acolor to be extracted as the origin so as to generate the key signal.However, this invention is not limited to such embodiment. For example,each pixel of the foreground picture may be expressed by a color spacewith reference to the color signal levels of red, blue and green,instead of the YUV color space. In this manner, by connecting the deviceof this invention to various equipments for processing image signals bythese color signals, the key signal may be generated by a simplestructure.

In addition, in the above-described embodiment, the key signal for colorcancel processing is generated in addition to the key signal for chromakey. However, this invention is not limited to such embodiment, and mayalso be applied broadly to the case where only the key signal for chromakey is generated or the case where other key signals for editing aregenerated.

Also, in the above-described embodiment, the key signal is generated andpicture synthesis is carried out by using the generated key signal.However, this invention is not limited to such embodiment, and may alsobe applied broadly to the case where the generated key signal isoutputted to a switcher or the like.

Moreover, in the above-described embodiment, continuous picturesconstituted by image signals are processed by chroma key processing.However, this invention is not limited to such embodiment, and may alsobe applied broadly to the case where still pictures are processed bychroma key processing.

Also, in the above-described second embodiment, the conditions forchroma key processing are automatically set by correcting theneighboring four representative points by an equal distance. However,this invention is not limited to such embodiment, and the distance mayalso be correct by carrying out weighting in accordance with the anglefrom a pixel clicked by the user. That is, as shown in FIG. 36, in thecase where a pixel clicked by the user internally divides theneighboring four representative points by angles θ1, θ2, φ1 and φ2, thedistance of each representative point is corrected by calculatingrespective values xa, xb, xc and xd so that the distance r (φ, θ)corresponds to the distance d detected at step SP41 in accordance withthe following arithmetic processing, thus enabling weighting to changethe representative points. In this manner, the conditions for chroma keyprocessing may be automatically set further in detail. $\begin{matrix}{{r\left( {\varphi,\theta} \right)} = \frac{S}{\left( {{\theta 1} + {\theta \quad 2}} \right) \cdot \left( {{\varphi 1} + {\varphi \quad 2}} \right)}} & (22)\end{matrix}$

S=(roa+x)·θ2·φ2+(rob+x)·θ1·φ2+(roc+x)·θ2·φ1+(rod+x)·θ1·φ1

xa:xb=θ2:θ1

xa:xc=φ1:φ2

xc:xd=θ2:θ1

Also, in the above-described second embodiment, designation by the useris accepted with respect to a portion to be replaced with the backgroundand a portion not to be replaced with the background. However, thisinvention is not limited to such embodiment, the designation by the usermay be accepted only with respect to either one portion, or may beaccepted with respect to a range setting the key signal to a value of 0to 1.

In addition, in the above-described embodiment, the reference planeperpendicular to the Y-axis with reference to the center color is set soas to form the first to third display sections, in the vector scopedisplay section. However, this invention is not limited to suchembodiment, and may also be applied broadly to the cases where variousforms of displays are made, for example, the case where the first tothird display sections are formed by a reference plane with reference tothe origin of the UVY axes.

Moreover, in the above-described embodiment, the inner sphere and theouter sphere are displayed by lines in the vector scope display section.However, this invention is not limited to such embodiment, and an areawithin the inner sphere, an area within the outer sphere and an areabetween the inner and outer spheres may be displayed by displaying anarea corresponding to the value of the key signal.

In addition, in the above-described embodiment, the display is updatedevery time the representative point and the center color are changed inthe vector scope display section. However, this invention is not limitedto such embodiment. In the vector scope display section, updateprocessing may be omitted by enabling grasp of the general relationbetween the inner sphere, the outer sphere and the foreground picture.

Also, in the above-described embodiment, the key signal characteristicsetting screen is formed by the color space display section and thevector scope display section. However, this invention is not limited tosuch embodiment, and may also be applied broadly to the case where thekey signal characteristic setting screen is formed in various forms, ifnecessary.

Also, though in the above-described embodiment, the boundary for chromakey processing is expressed by 26 points, this invention is not limitedto such embodiment. For example, in the case where points are set at aconformal spacing of 30 degrees or in the case where points are set at anon-conformal spacing, the boundary may be expressed by various numbersof points, if necessary.

In addition, in the above-described embodiment, the boundary for chromakey processing is set by double substantially spherical shapes. However,this invention is not limited to such embodiment, and may also beapplied to the case where triple boundaries are set so that thecharacteristic of chroma key processing is set further in detail withreference to the intermediate boundary.

Also, in the above-described embodiment, in the case where each pixel ofthe foreground picture is expressed by the YUV color space having thecenter color of a color to be extracted as the origin so as to generatethe key signal, distribution of pixels of the foreground picture isdisplayed in the vector scope display section. However, this inventionis not limited to such embodiment, and may also be applied broadly tothe case where each pixel of the foreground picture is expressed by acolor space with reference to color signal levels of red, blue, andgreen, and the case where the foreground picture is expressed andprocessed on the UV two-dimensional plane similar to that of theconventional technique.

What is claimed is:
 1. A key signal generating device for generating akey signal for synthesizing a foreground picture and a backgroundpicture, the device comprising: polar coordinate generating means fordetecting a polar coordinate of each pixel of the foreground picture ina three-dimensional color space having a center color of a color to beextracted from the foreground picture as an origin; and key signalgenerating means for generating the key signal in accordance with thedistance of the polar coordinate from the origin.
 2. The key signalgenerating device as claimed in claim 1, wherein the key signalgenerating means sets a boundary to surround the origin in the colorspace and generates the key signal in accordance with the position ofeach pixel with respect to the boundary from the distance.
 3. The keysignal generating device as claimed in claim 2, wherein the key signalgenerating means prescribes the boundary by a group of points set at apredetermined spacing in the color space, carries out interpolation ofpolar coordinate values of the group of adjacent points so as to detectthe distance of the boundary from the origin corresponding to the polarcoordinate value of the pixel, and generates the key signal withreference to the distance of the boundary.
 4. The key signal generatingdevice as claimed in claim 2, wherein the key signal generating meansforms the boundary in double, sets the key signal to allocate thebackground picture, in the case where the pixel is located on the innerside of an inner boundary, sets the key signal to allocate theforeground picture, in the case where the pixel is located on the outerside of an outer boundary, and sets the key signal to carry outweighting and addition of the foreground picture and the backgroundpicture and allocate the foreground picture and the background picturein accordance with the position of the pixel, in the case where thepixel is located between the inner boundary and the outer boundary. 5.The key signal generating device as claimed in claim 2, wherein the keysignal generating means has a look-up table formed therein at everypredetermined angle in the color space, the look-up table having thedistance to the boundary recorded thereon, and generates the key signalby accessing the look-up table using the angle of each pixel in thecolor space as an address.
 6. The key signal generating device asclaimed in claim 4, wherein the key signal generating means has alook-up table on which the distance to the boundary is recorded, atevery predetermined angle in the color space, and generates the keysignal by accessing the look-up table using the angle of each pixel inthe color space as an address, the look-up table being formed bydistance information about the distance to the inner or outer boundary,and gain information indicating a change in value of the key signal perunit distance between the inner boundary and the outer boundary.
 7. Thekey signal generating device as claimed in claim 1, wherein the keysignal generating means detects the distance and angle in atwo-dimensional space, formed by projecting a line connecting the originand each pixel on a predetermined plane on the color space, and thendetects a key signal in accordance with the distance in the color spaceon the basis of the distance and angle in the two-dimensional space. 8.The key signal generating device as claimed in claim 1, wherein thecolor space is constituted by a color space using a luminance signallevel and color-difference signal levels as reference axes.
 9. The keysignal generating device as claimed in claim 1, further comprising colorcancel key signal generating means for generating a key signal for colorcancel processing for restraining the signal level of the foregroundpicture, the color cancel key signal generating means detecting a polarcoordinate of each pixel of the foreground picture in a two-dimensionalcolor plane with reference to color-difference signal levels having acenter color to be color-cancelled as an origin, the color cancel keysignal generating means generating the key signal for color cancelprocessing in accordance with the distance of the polar coordinate fromthe origin.
 10. A key signal generating method for generating a keysignal for synthesizing a foreground picture and a background picture,comprising the steps of; setting a value of the key signal in accordancewith the distance from a predetermined reference color to each pixel ofthe foreground picture in a three-dimensional color space, displaying apicture as a result of processing by the key signal and a characteristicsetting screen for the key signal, and changing characteristics of thekey signal using information obtained through the characteristic settingscreen, and the picture as a result of processing.
 11. The key signalgenerating method as claimed in claim 10, wherein the foregroundpicture, the background picture, and a picture of the key signal aredisplayed together with the picture as a result of processing.
 12. Thekey signal generating method as claimed in claim 10, wherein thecharacteristic setting screen includes a picture formed by viewing, froma desired viewpoint, a three-dimensional color space in which a patternindicating pixels of the foreground picture and/or the characteristicsof the key signal is arranged.
 13. The key signal generating method asclaimed in claim 10, wherein the characteristic setting screen includesa picture formed by projecting, onto a predetermined reference plane, athree-dimensional color space in which a pattern indicating pixels ofthe foreground picture and/or the characteristics of the key signal isarranged.
 14. A key signal generating method for generating a key signalfor synthesizing a foreground picture and a background picture,comprising the steps of; setting a value of the key signal in accordancewith the distance of each pixel of the foreground picture from areference center color in a three-dimensional color space, anddisplaying position information of a pixel designated in the foregroundpicture from a predetermined reference position in the three-dimensionalcolor space.
 15. The key signal generating method as claimed in claim14, wherein the reference position is the position of the center color,the position information including the distance and direction from thecenter color.
 16. The key signal generating method as claimed in claim15, wherein information for key signal setting is accepted in a formatcorresponding to the position information, and conditions for generatingthe key signal are set in accordance with the accepted information. 17.The key signal generating method as claimed in claim 15, wherein aboundary surrounding the center color is expressed by a group of pointsset in the three-dimensional color space so as to set a value of the keysignal in accordance with each pixel of the foreground picture withreference to the boundary, and a polar coordinate value of at least onepoint of the group of points is corrected in accordance with theposition information so as to set conditions for generating the keysignal in accordance with the position information.
 18. The key signalgenerating method as claimed in claim 14, wherein information for keysignal setting is accepted in a format corresponding to the positioninformation, and conditions for generating the key signal are set inaccordance with the accepted information.
 19. A key signal generatingmethod for generating a key signal for synthesizing a foreground pictureand a background picture, comprising the step of; projecting a pixelforming the foreground picture on a reference plane set on athree-dimensional color space so as to display the position of the pixelforming the foreground picture on the three-dimensional color space. 20.The key signal generating method as claimed in claim 19, wherein thereference plane is constituted by a plane perpendicularly crossing areference axis of luminance level in the three-dimensional color space,and is set at every predetermined luminance level, the key signalgenerating method segmenting pixels of the foreground picture withreference to the luminance level of each reference plane so as toselectively project the pixel of each segment onto the correspondingreference plane.
 21. The key signal generating method as claimed inclaim 20, wherein each pixel is displayed by projecting onto thereference plane in accordance with the hue of each pixel and theluminance level of the corresponding reference plane.
 22. The key signalgenerating method as claimed in claim 20, wherein a value of the keysignal is set in accordance with the distance from a predeterminedreference color on the three-dimensional color space.
 23. The key signalgenerating method as claimed in claim 22, wherein an area correspondingto the key signal is displayed on the reference plane.
 24. The keysignal generating method as claimed in claim 23, wherein a change of thearea corresponding to the value of the key signal on the reference planeis accepted by operating a pointing device, and the value of the keysignal with respect to the distance from the reference color is changedin response to the change.
 25. The key signal generating method asclaimed in claim 19, wherein a value of the key signal is set inaccordance with the distance from a predetermined reference color on thethree-dimensional color space.
 26. The key signal generating method asclaimed in claim 25, wherein an area corresponding to the value of thekey signal is displayed on the reference plane.
 27. The key signalgenerating method as claimed in claim 26, wherein a change of the areacorresponding to the value of the key signal on the reference plane isaccepted by operating a pointing device, and the value of the key signalwith respect to the distance from the reference color is changed inresponse to the change.
 28. The key signal generating method as claimedin claim 19, wherein each pixel is displayed by projecting onto thereference plane in accordance with the hue of each pixel and theluminance level of the corresponding reference plane.
 29. A key signalgenerating method for generating a key signal for synthesizing aforeground picture and a background picture, comprising the steps of;setting a value of the key signal in accordance with the distance from apredetermined reference color in a three-dimensional color space, anddisplaying a picture viewed from a desired viewpoint using each pixel ofat least the foreground picture which is located at a correspondingposition on the three-dimensional color space.
 30. The key signalgenerating method as claimed in claim 29, wherein the position of theviewpoint is changed in response to the operation of an operatingelement.
 31. The key signal generating method as claimed in claim 30,wherein an area corresponding to the value of the key signal isdisplayed in the picture.
 32. The key signal generating method asclaimed in claim 31, wherein a change of the area is accepted on thepicture.
 33. The key signal generating method as claimed in claim 29,wherein a reference color expressing the hue in the three-dimensionalcolor space is displayed in the picture.
 34. The key signal generatingmethod as claimed in claim 29, wherein a reference axis in thethree-dimensional color space is displayed in the picture.
 35. A keysignal generating method for generating a key signal for synthesizing aforeground picture and a background picture, the method comprising: astep of detecting a polar coordinate of each pixel of the foregroundpicture in a three-dimensional color space having a center color of acolor to be extracted from the foreground picture as an origin; and astep of generating the key signal in accordance with the distance of thepolar coordinate from the origin.
 36. A picture synthesizing device forsynthesizing a foreground picture and a background picture in accordancewith a key signal, the key signal being generated by using polarcoordinate generating means for detecting a polar coordinate of eachpixel of the foreground picture in a three-dimensional color spacehaving a center color of a color to be extracted from the foregroundpicture as an origin, and key signal generating means for generating thekey signal in accordance with the distance of the polar coordinate fromthe origin.
 37. A picture synthesizing device for synthesizing aforeground picture and a background picture in accordance with a keysignal from a key signal generating section, the key signal generatingsection setting a value of the key signal in accordance with thedistance from a predetermined reference color to each pixel of theforeground picture in a three-dimensional color space, displaying apicture as a result of processing by the key signal and a characteristicsetting screen for the key signal, and changing characteristics of thekey signal by information obtained through the characteristic settingscreen, and changing the picture as a result of processing.
 38. Apicture synthesizing device for synthesizing a foreground picture and abackground picture in accordance with a key signal from a key signalgenerating section, the key signal generating section setting a value ofthe key signal in accordance with the distance of each pixel of theforeground picture from a reference center color in a three-dimensionalcolor space, and displaying position information of a pixel designatedin the foreground picture from a predetermined reference position in thethree-dimensional color space.
 39. A picture synthesizing device forsynthesizing a foreground picture and a background picture in accordancewith a key signal from a key signal generating section, the key signalgenerating section projecting a pixel forming the foreground pictureonto a reference plane set on a three-dimensional color space so as todisplay the position of the pixel forming the foreground picture on thethree-dimensional color space.
 40. A picture synthesizing device forsynthesizing a foreground picture and a background picture in accordancewith a key signal from a key signal generating section, the key signalgenerating section setting a value of the key signal in accordance withthe distance from a predetermined reference color in a three-dimensionalcolor space, and locating each pixel of at least the foreground pictureat a corresponding position on the three-dimensional color space so asto display a picture viewed from a desired viewpoint.
 41. An editingdevice for synthesizing a foreground picture and a background picture inaccordance with a key signal from a key signal generating section,editing a plurality of frames including synthesized frames, andoutputting the edited frames, the key signal generating sectionincluding polar coordinate generating means for detecting a polarcoordinate of each pixel of the foreground picture in athree-dimensional color space having a center color of a color to beextracted from the foreground picture as an origin; and key signalgenerating means for generating the key signal in accordance with thedistance of the polar coordinate from the origin.
 42. An editing devicefor synthesizing a foreground picture and a background picture inaccordance with a key signal from a key signal generating section,editing a plurality of frames including synthesized frames, andoutputting the edited frames, the key signal generating section settinga value of the key signal in accordance with the distance from apredetermined reference color to each pixel of the foreground picture ina three-dimensional color space, displaying a picture as a result ofprocessing by the key signal and a characteristic setting screen for thekey signal, and changing characteristics of the key signal byinformation obtained through the characteristic setting screen, andchanging the picture as a result of processing.
 43. An editing devicefor synthesizing a foreground picture and a background picture inaccordance with a key signal from a key signal generating section,editing a plurality of frames including synthesized frames, andoutputting the edited frames, the key signal generating section settinga value of the key signal in accordance with the distance of each pixelof the foreground picture from a reference center color in athree-dimensional color space, and displaying position information of apixel designated in the foreground picture from a predeterminedreference position in the three-dimensional color space.
 44. An editingdevice for synthesizing a foreground picture and a background picture inaccordance with a key signal from a key signal generating section,editing a plurality of frames including synthesized frames, andoutputting the edited frames, the key signal generating sectionprojecting a pixel forming the foreground picture onto a reference planeset on a three-dimensional color space so as to display the position ofthe pixel forming the foreground picture on the three-dimensional colorspace.
 45. An editing device for synthesizing a foreground picture and abackground picture in accordance with a key signal from a key signalgenerating section, editing a plurality of frames including synthesizedframes, and outputting the edited frames, the key signal generatingsection setting a value of the key signal in accordance with thedistance from a predetermined reference color in a three-dimensionalcolor space, and locating each pixel of at least the foreground pictureat a corresponding position on the three-dimensional color space so asto display a picture viewed from a desired viewpoint.